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Executive Summary 


This purpose of this report is to document the Applied Meteorology Unit's (AMU) evaluation of 
the National Centers for Environmental Prediction (NCEP) 29-km eta model. NCEP started running 
the 29-km version of the eta (meso-eta) model in August 1995 and they continue to update the model 
configuration, initialization, and physical parameterizations. The objective of the evaluation is to 
assess the utility of the meso-eta model for local weather forecasting in support of 45th Weather 
Squadron (45WS), Spaceflight Meteorology Group (SMG), and National Weather Service (NWS) 
Melbourne (MLB) operational requirements. 

The evaluation protocol was determined by a technical group comprised of meteorologists and 
forecasters from 45WS, SMG, NWS MLB, and AMU. The evaluation protocol consisted of both 
objective and subjective components. The objective verification of the meso-eta model focused on the 
overall accuracy of wind, temperature, and moisture forecasts at selected stations. The statistical 
measures used to quantify model forecast errors were the bias, root mean square (RMS) error, 
standard deviation, and consistency. Using these statistics, point forecasts from the meso-eta model 
were verified against standard surface and upper air observations. The subjective evaluation 
includes warm-season forecast exercises and phenomenological verification focusing primarily on 
limited case studies and seasonal evaluations of sea breezes, thunderstorms, and cold fronts. 
Subjective verification is performed using gridded data only from 0300 UTC model runs. 

Objective verification results generally indicate that meso-eta model point forecasts at selected 
stations exhibit minimal error growth in terms of RMS errors, and except for a few parameters, are 
reasonably unbiased. However, there are some exceptions identified in this evaluation as indicated 
by the following list of model biases. 

• 2-m dew point temperatures and 10-m wind speeds are typically overestimated 
at Cape Canaveral Air Station, FL (XMR) during the cool season. 

• Diurnal changes exist in the average forecast errors for 2-m temperature, 10-m 
wind speed and MSLP at Edwards Air Force Base, CA (EDW). 

• At XMR and Tampa Bay, FL (TBW), warm (cool) season forecast soundings are 
typically drier and more stable (unstable) than observed. 

• The height of the lower tropospheric inversion at XMR and TBW is 
misrepresented during the cool season. 

• Tropopause heights are misrepresented by the model at all three stations. 

Results from the objective verification do not indicate whether the model is more accurate overall 
during either the warm or cool season. However, results from the subjective verification suggest that 
the model forecasts over central Florida may be more useful during the cool season. This statement is 
based on the fact that the meso-eta model resolution is not yet sufficient to resolve the small-scale 
details of sea and river/lake breeze circulations, thunderstorm outflow boundaries, and other 
phenomena which play a dominant role in determining the short-term evolution of weather over east 
central Florida during the warm season. 

Objective verification results also demonstrate that forecasts of selected parameters are reliable 
over the course of an entire season. On the other hand, results from the subjective verification 
demonstrate that model forecasts of developing weather events such as thunderstorms, sea breezes, 
cold fronts, etc. are not always as accurate as implied by the seasonal error statistics. The subjective 
verification is indeed very important to quantify added value of model forecasts for these specific 
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phenomena which can not be readily inferred from statistics over many cases (i.e. from objective 
verification). Some results from the subjective evaluation which can be important for operational 
forecast concerns include the following. 

• Sea-breeze case studies reveal that the model generates a dynamically-consistent 
thermally direct circulation over the Florida peninsula, although at a larger scale 
than observed. Seasonal verification of sea breezes indicates that the model 
forecasts the occurrence of east and/ or west coast sea breezes roughly 50% of the 
time they are observed. 

• Thunderstorm verification reveals that the meso-eta model is capable of 
predicting areas of organized convection, particularly during the late afternoon 
hours. On the other hand, the model is also subject to subtle errors that can lead 
to incorrect forecasts of warm season convective precipitation. In particular, the 
model often generates excessive rainfall during the morning hours and is not 
capable of accurately forecasting individual thunderstorms. 

• Verification of cold fronts during the cool season reveals that the model is 
capable of forecasting a majority of cold frontal passages through east central 
Florida to within ±l-h of observed frontal passage. 

Warm season forecast exercises demonstrate that animation of 3-h model output with color 
enhancements and overlay of multiple fields is helpful to identify features and trends that could 
become important for developing weather. Availability of digital gridded model output at 3-h 
intervals is important because it gives users the flexibility in display and analysis options while 
providing the temporal resolution needed to track specific aspects of forecast weather events such as 
the timing of cold frontal passages. In order to exploit the four-dimensional capability of the meso- 
eta and other models in forecasting possible realizations of the atmosphere, sufficient communication 
bandwidth and computer processing power are necessary to retrieve, process, and examine output 
data. This requirement will become more important in the future as NCEP increases the number of 
meso-eta model runs per day, the model resolution, and potentially the frequency of model output. 

The evaluation methodology used in this study enables an assessment of both forecast accuracy 
and utility for model users. However, the short 4-month evaluation periods impose limitations on 
the completeness of the verification, particularly with regard to sample sizes. Subjective verifications 
in this evaluation are limited to simple case studies in order to examine a greater number of 
phenomenological features which impact 45WS, SMG, and NWS MLB operations. The case studies 
highlight some of the capabilities and limitations of the meso-eta model but are not designed to fully 
explore the cause(s) of model errors. Seasonal evaluations of sea breezes, thunderstorms, and cold 
fronts are designed to quantify how consistently the model provides utility in forecasts of these 
phenomena. However, more detailed examination of model capabilities and limitations is beyond 
the scope of the present evaluation given available AMU resources. 

Ongoing changes to the model configuration, initialization, and physical parameterizations 
could modify some of the seasonal error characteristics identified by the AMU's objective verification 
of surface and upper air forecasts. In order to increase the sample size and track possible changes in 
model accuracy, the objective component of the meso-eta evaluation is being extended to include a 
second warm and cool season period from May through August 1997 and October 1997 through 
January 1998, respectively. A comparison between results from the 1996 and 1997 seasons will 
highlight any changes in the error characteristics at selected stations which may occur in response to 
updates in the meso-eta model configuration. This analysis will also prove useful for model users 
since the 1997/1998 results will be more representative of the meso-eta model's current capabilities. 
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1.0 Introduction 


1.1 Background 

Weather support for ground and aerospace operations at the Kennedy Space Center (KSC) and 
Cape Canaveral Air Station (CCAS) requires accurate forecasts of winds, clouds, ceilings, fog, rain, 
lightning, and visibility. The National Aeronautics and Space Administration (NASA) and United 
States Air Force (USAF) funded private corporations to develop and deliver numerical weather 
prediction systems that are designed to run locally on high performance workstations. In general, the 
objective of installing and running local mesoscale modeling systems is to produce more precise and 
detailed short-range (< 24 h) forecasts than operational models such as the Nested Grid Model 
(NGM) and eta model run at the National Centers for Environmental Prediction (NCEP). Local 
modeling systems have the potential to provide added value for short-range forecasting because they 
typically incorporate local data, run at finer horizontal resolutions over smaller domains, and use 
more sophisticated physical parameterizations than national-scale, operational models. 

The main components of the local mesoscale modeling systems developed for KSC /CCAS are the 
Mesoscale Atmospheric Simulation System (MASS) and Regional Atmospheric Modeling System 
(RAMS) models. The RAMS model is used in the Emergency Response Dose Assessment System 
(ERDAS) and the Parallelized RAMS WEather Simulation System (PROWESS). MASS is designed to 
provide forecasts of specific thunderstorm-related phenomena such as precipitation and high winds. 
ERDAS is designed for range safety to provide emergency response guidance for operations at 
KSC /CCAS in case of a hazardous material release or aborted vehicle launch. PROWESS is designed 
to forecast the development of site-specific thunderstorms over the KSC/CCAS work areas. 

As part of their ongoing evaluation and technology transition plan, the Applied Meteorology 
Unit (AMU) performed detailed evaluations of MASS (Manobianco 1996) and ERDAS (Evans 1996). 
Because ERDAS has mesoscale modeling and diffusion capabilities that are far more advanced than 
current local models, it was found to provide significant added value for range safety personnel. 
Similar high resolution guidance designed specifically for KSC/CCAS safety applications is not 
routinely available from other operational centers. In response to its proven utility and operational 
need, ERDAS is currently being transitioned into KSC/CCAS safety operations as part of the 
replacement and upgrade of the Meteorological And Range Safety Support (MARSS) system. On the 
other hand, larger-scale forecasts of wind, temperature, precipitation, and other parameters for many 
areas including east-central Florida are routinely provided by the NCEP. Many components of the 
MASS evaluation indicated that it could not consistently produce more skillful short-range 
precipitation and wind forecasts than those already provided by the NCEP's NGM model. 

When the AMU completed the MASS evaluation in December 1995, two subsequent 
teleconferences were convened with NASA Headquarters, NASA KSC, 45th Weather Squadron 
(45WS), Spaceflight Meteorology Group (SMG), and the National Weather Service (NWS) Melbourne 
(MLB) to review the evaluation results and discuss options for an out-of-cycle or "mid-course 
correction" to the AMU mesoscale modeling task. The following points were raised during these two 
teleconferences. 

• It was evident from the evaluation that the current version of MASS did not 
provide sufficient added value over NCEP models to justify the cost of 
continuing the evaluation with the intent to transition MASS for operational use. 

• Data deficiencies such as limited access to NCEP gridded data and no access to 
digital data from the WSR-88D at MLB and the KSC/CCAS network of five 915- 
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MHz Doppler Radar Wind Profilers (DRWP) probably limited the utility of local 
modeling systems such as MASS and PROWESS. 

• NCEP started running a 29-km or mesoscale version of the eta model in August 
1995 (Mesinger 1996) and plans to run a 10-km, non-hydrostatic version of the 
model over the entire United States by the year 2000 (Kalnay et al. 1996). 

The first two points suggest that a number of issues need to be addressed before local modeling 
systems such as MASS and PROWESS could provide significant added value to justify their life-cycle 
costs. In addition, 45WS, SMG, and NWS MLB agreed that an evaluation of the 29-km or meso-eta 
model would likely result in a low-risk payoff within one year, namely that the AMU would be able 
to determine the utility of NCEP's best mesoscale model for local forecasting. Based on these points, 
45WS, SMG, and NWS MLB reached a consensus to shift the AMU modeling task from evaluating 
local models to evaluating NCEP's operational mesoscale model. As a result of this "mid-course 
correction", the AMU was directed to terminate all work with MASS, delay any prioritized work on 
evaluating PROWESS, and begin evaluating the meso-eta model. 

1.2 Applied Meteorology Unit Tasking 

Under the Mesoscale Modeling Task (005), Subtask 2, the AMU evaluated the most effective ways 
to use NCEP's 29-km eta model to meet 45WS, SMG and NWS MLB requirements. The evaluation 
methodology was determined by a technical working group consisting of several meteorologists and 
forecasters from the AMU, 45WS, SMG, and NWS MLB. Based on recommendations from the 
technical working group, the AMU determined the data acquisition requirements, and designed and 
implemented the evaluation protocol. 

1.3 Purpose and Organization of the Report 

This purpose of this report is to document the AMU's evaluation of the 29-km eta model. Section 
2 provides a brief overview of the eta model. The evaluation protocol is summarized in Section 3. 
Section 4 discusses results of the objective evaluation while Section 5 discusses results from the 
subjective evaluation. A summary and lessons learned from the overall evaluation are presented in 
Section 6. Finally, Section 7 highlights the AMU's future work on evaluating the 29-km eta model. 


2 



2.0 Eta Model Overview 


The primary mesoscale modeling efforts at NCEP are focused on the development of the eta 
model (Rogers et al. 1995). The eta model takes its name from a vertical coordinate that is a 
generalization of the terrain-following sigma coordinate. However, the eta coordinate is normalized 
by sea-level pressure rather than surface pressure so the resulting eta surfaces are quasi-horizontal. 
This feature of the eta coordinate system eliminates errors in pressure gradient terms computed over 
steeply sloping terrain and also produces model orography in the form of discrete blocks or steps 
(Mesinger and Black 1992). 

The original version of the eta model with a horizontal resolution of 80 km and 38 vertical layers 
replaced the Limited-Area Fine Mesh model in June 1993 (Black 1994). In October 1995, NCEP 
increased the horizontal resolution of the operational "early" eta model from 80 km to 48 km. The 
term "early" refers to the fact that the 48-km eta model run begins 1.25 h after the data cutoff time to 
provide forecast guidance as quickly as possible (Mesinger 1996). Since August 1995, NCEP has also 
been running a mesoscale version of the eta model with a horizontal resolution of 29 km and 50 
vertical layers (Mesinger 1996). The 29-km or meso-eta model is started after the 48-km model run 
has completed. 

The relevant numerics and physics of the eta model are summarized in Table 2.1. The specific 
details regarding the dynamics, physics, horizontal domain, initialization, and other aspects of the eta 
model configuration are provided by Black (1994) and Rogers et al. (1995, 1996). Following upgrades 
to the 48-km version in January 1996, the "early" and meso-eta model configurations are now 
identical except for resolution and data assimilation. The Eta Data Assimilation System (EDAS) for 
the "early" eta consists of four 3-h analysis-forecast cycles that incorporate high frequency 
observations such as aircraft and profiler data (Rogers et al. 1996). The meso-eta is also intialized 
with the EDAS but it runs for one 3-h analysis-forecast cycle from 0000-0300 UTC or 1200-1500 UTC. 
The EDAS is currently started with a first guess from the Global Data Assimilation System (GDAS). 
In the near future, NCEP plans to restructure the EDAS so that it uses the previous EDAS cycle rather 
than the GDAS to provide a first guess for subsequent analyses. 


Table 2.1. Eta model attributes*. 


Dynamics 

Model top = 25 mb 

Time step = 72 s 

Semi-staggered Arakawa E-grid 

Gravity wave coupling scheme 

Silhouette-mean orography 

Split-explicit time differencing 

Physics 

Explicit grid-scale cloud and precipitation 

Modified Betts-Miller convective adjustment 

Mellor-Yamada (2.5) for free atmosphere vertical turbulent exchange 
Mellor-Yamada (2.0) near ground 

Geophysical Fluid Dynamics Laboratory radiation scheme 

Viscous sublayer over water 


*from Black (1994) and Rogers et al. (1996) 
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3.0 Evaluation Protocol 


The evaluation protocol was determined by a technical group consisting of several meteorologists 
and forecasters from 45WS, SMG, NWS MLB, and the AMU. The technical working group held three 
teleconferences that resulted in an evaluation strategy consisting of both objective and subjective 
components. The objective verification of the eta model focuses on the overall accuracy of wind, 
temperature, and moisture forecasts at selected stations. The subjective or phenomenological 
verification assesses the eta model's capabilities in forecasting convective activity, the location and 
movement of fronts and the onset, depth, and propagation of sea-breezes. All of these parameters are 
important in evaluating launch commit criteria (LCC) for manned and unmanned vehicle launches 
and flight rules (FR) for Shuttle landings. The following sections summarize how the AMU is 
currently accessing the 29-km eta model forecasts and provide specifics of the objective and 
subjective evaluation criteria designed by the technical working group. 

3.1 Data Acquisition 

The 29-km eta data are obtained via the Internet from NOAA's Information Center (NIC) FTP 
server. The gridded data from the 0300 UTC cycle of the meso-eta are downloaded at approximately 
0900 UTC. (Note the gridded data from the 1500 UTC cycle of the eta model are not being used for 
the evaluation discussed here.) NCEP interpolates the 29-km eta model output to the Advanced 
Weather Interactive Processing System (AWIPS) 40-km grid at 3-h intervals for the entire meso-eta 
33-h forecast period. After the meso-eta data on the AWIPS grid are retrieved from the NIC, a subset 
of the data are decoded from GRIdded Binary (GRIB) format using software contained in the GEneral 
Meteorological PAcKage (GEMPAK; desJardins et al. 1997). In addition to gridded data, the point 
forecasts at selected stations are downloaded from both the 0300 UTC and 1500 UTC cycles of the 
meso-eta model. These files contain surface and upper air parameters at 1-h intervals and are 
decoded from Binary Universal Form for the Representation of meteorological data (BUFR) using 
GEMPAK software. 

3.2 Objective Evaluation Criteria 

The objective verification of the 29-km eta model examines forecast errors for the parameters 
shown in Table 3.1. The station or point forecasts from the 0300 UTC and 1500 UTC meso-eta model 
cycles are verified against standard surface and rawinsonde observations. Hourly surface 
observations are taken at the Shuttle Landing Facility, FL (TTS), Edwards Air Force Base, CA (EDW), 
and Tampa International Airport, FL (TPA). Rawinsonde observations are taken twice daily at EDW, 
CCAS (XMR), and Tampa Bay (TBW). The XMR and EDW stations are selected because they are the 
primary and secondary landing sites for the Shuttle. The TBW site is chosen to compare model 
errors at two coastal stations on the eastern (XMR) and western (TBW) edge of the Florida peninsula. 
Station forecasts are extracted from the meso-eta model grid point nearest to the rawinsonde 
observation sites. Although surface and rawinsonde observations are not co-located at XMR and 
TBW, the available sites are separated by not more than about 30 km (i.e. the meso-eta model grid 
spacing). In order to avoid confusion, all subsequent references to rawinsonde and surface 
verification will use the rawinsonde station identifiers (XMR, TBW, EDW). 

Hourly surface observations are used to verify 0- to 33-h forecasts of mean sea-level pressure, 2-m 
temperature (T) and dew point temperature (T d ), and 10-m winds (u, v). Upper air winds (u, v), 
temperature (T), moisture (q), and height (z) are verified at 25-mb intervals from 1000 to 50 mb and at 
selected height levels from 1000 to 70000 ft. Log-linear interpolation of data is used between available 
pressure levels while straight linear interpolation is used between available height levels. At XMR 
and EDW, rawinsondes are usually released at 0900 (1000) UTC and 2100 (2200) UTC during daylight 
savings (standard) time. However, variations in the release time do occur so three verification 
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periods are defined according to the schematic illustrated in Fig. 3.1. Periods 1, 2, and 3 range from 5- 
10 h, 17-22 h, and 29-33 h into the forecast cycle, respectively. Although rawinsonde release times 
may vary at XMR and EDW, forecast verification at all stations always coincides with the observation 
time. At TBW, forecast verification occurs at 0000 and 1200 UTC. 


Table 3.1. Meso-eta objective verification parameters. 


Parameter 

Levels 

Mean sea-level pressure 

— 

u, v 

10 m 

T, T d 

2m 

u, v, T, q, z 

selected* 

Precipitable water (PWAT; mm) 

— 

Convective available potential energy (CAPE; J kg’ 1 ) 

— 

Convective inhibition (CINS; J kg' 1 ) 

— 

Lifted index (LIFT; *) 

— 

K index (KINX) 

— 

Ground relative helicity (HLCY; m 2 s’ 2 ) 

0-3 km 

Microburst day potential index (MDPI) 

— 

Thickness 

1000-850 mb ' 

Mean layer wind 

850-500 mb 

Mean layer relative humidity 

850-500 mb 


*refer to text for description 
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Figure 3.1. Definition of available windows for objective verification of upper air forecasts. 

The parameters such as precipitable water, mean layer wind and relative humidity, and stability 
indices are included because they can be used to assess the potential for convective activity. Except 
for helicity (Davies-Jones et al. 1990; Lilly 1986) and Microburst Day Potential Index (MDPI; Wheeler 
and Roeder 1996), stability indices are derived using routines in GEMPAK. Since convection is 
usually not a forecasting concern at EDW, stability indices are not presented for that station. Winds 
are verified in terms of speed and direction since these quantities are more familiar to model users 
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than zonal ( u ) and meridional (v) wind components. Except for the 10-m winds, the meso-eta model 
forecasts are not benchmarked against other models, climatology, or persistence. For 10-m winds, 
model forecasts are compared with persistence for a period of 1 to 6 h. 

The objective verification considers 29-km eta model forecasts from May through August 1996 
(warm season) and from October 1996 through January 1997 (cool season). Within each seasonal 
evaluation period, the original evaluation protocol specified that verification would be stratified by 
average wind direction in the layer from 950 to 600 mb using seven wind regimes following Holle et 
al. (1992). This type of stratification is designed to determine if the model forecast errors are sensitive 
to specific flow regimes which are defined by layer-averaged wind directions. However, there are a 
limited number of available forecast/ observation pairs in each of the seven wind regimes during the 
four-month warm or cool season. These small sample sizes make it difficult to draw meaningful 
conclusions regarding error characteristics as a function of flow regime (Panofsky and Brier 1958). 
Instead, westerly and easterly regimes are defined based on winds that are approximately 
perpendicular to Florida's east coast. In particular, the 950- to 600-mb layer-averaged winds are 
considered westerly between 158° and 338“ and easterly outside that range. Typical sample sizes for 
surface and upper air forecast/observation pairs for each station and wind regime are listed in Table 
3.2. 


Table 3.2. Typical seasonal sample sizes for valid pairs of forecast /observed point data. 
The actual sample sizes vary for any given verification time, vertical level, and parameter but 
are on the order of the numbers shown here. "West" and "East" represent sample sizes after 
stratification by the 950- to 600-mb layer-averaged wind direction while "All" represents the 
size of combined data sets (see text). Only data from 0300 UTC forecasts/observations are 
included in the surface data sets (see section 4.1). Upper air samples use all available data 
from both 0300 and 1500 UTC forecasts/observations. 



XMR 

Warm Cool 

TBW 

Warm Cool 

EDW 

Warm Cool 

All 

Surface Data West 

East 

70 70 

25 45 

45 25 

75 95 

30 45 

45 30 

75 70 

75 40 

0 30 

All 

Upper Air Data West 

East 

130 125 

65 90 

65 35 

145 145 

70 100 

75 45 

75 55 

65 40 

10 15 


The statistical measures used here to quantify model forecast errors are the bias, root mean 
square (RMS) error, standard deviation, and consistency. If O represents any of the variables in Table 
3.1, then forecast error is defined as O' = <t>, - o„ where the subscripts f and o denote forecast and 
observed quantities, respectively. In cases where the magnitude of the wind direction error exceeds 
180*, the error is recomputed by first subtracting 360° from the larger of the forecast or observed wind 
direction. The bias is computed as 


— 1 N 

bias (p, t) = O' = — X O' 
N j=i 


the RMSE is computed as 


RMSE (p,t) 


[MSE] l/2 =[ll(0') 2 

|_Nj = i 



and the standard deviation of the errors is computed as 


( 1 ), 


( 2 ), 
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( 3 ). 


Tin _ ,-i'« 

<r'(p.t) = — 1(0)' -O') 2 


When N is used rather than N-l in Equation (3), Murphy (1988) demonstrated that the following 
decomposition could be applied to the MSE: 


MSE = <£' 2 + <r /2 ( 4 ) 

Therefore, the total model error consists of contributions from model biases ( <f> /2 ) and random 
variations in the forecast or observed data (cr' 2 ). Note that if the model bias or systematic error is 
small, most of the MSE is due to random, non-systematic type variability. In fact, it is possible that a 
very small contribution to the non-systematic error could result simply from different reporting 
precision between the model forecasts and observations. 

In addition to these standard forecast verification methods, a fourth parameter is defined in an 
attempt to provide users with a measure of confidence in forecast stability. Following Leslie et al. 
(1994), model forecast consistency is computed at a point to measure the bias between subsequent 
forecasts verifying at the same time but initialized 12-h apart. If © represents any of the parameters 
from the 0300 UTC or 1500 UTC forecast cycles listed in Table 3.1, then consistency is defined as 

Consistency(p, t) [03 15] = j-y I [0 (O3 . I5] (*, * + 12) - © [15>03] (/ + 1. r)] 

f (5)- 

The subscript i in Equations (1) through (5) refers to an individual model forecast and N is the 
total number of point forecasts initialized at 0300 UTC and 1500 UTC. As with the forecast bias (Eq. 
1) model consistency is either positive or negative. Other non-standard measures which are 
independent of sign are defined by Leslie et al. (1994) but are not used in this study. On average, 
small values of consistency indicate that only minor differences occur between model runs. 
Conversely, larger values indicate that subsequent model runs tend to disagree with one another. 

Each of the statistics above are a function of pressure (p) and time (t). For quality control, gross 
errors in the data are screened manually and corrected, if possible. Individual errors which are 
greater than two standard deviations from the mean error (bias) are removed. Although this 
procedure removes approximately 5% of the data, it is very useful in objectively flagging bad data 
points. 

3.3 Subjective Evaluation Criteria 

The subjective component of the meso-eta model evaluation consists of daily, real-time warm 
season forecast exercises by AMU personnel and very limited analyses of selected case studies from 
both warm and cool seasons. GEMPAK/ntrans software is used on a daily basis to generate graphics 
and view animated loops of 3-hourly meso-eta model output. In this manner, forecast variables such 
as wind, moisture, stability indices, convergence, helicity, etc. are displayed using time series plots, 
vertical cross sections, horizontal sections, and time-height sections. At the end of each day, the 
AMU archived gridded forecasts of these variables from the 0300 UTC runs of the meso-eta model. 
Point forecasts were also collected from the 0300 UTC and 1500 UTC model runs at XMR, TBW, 
EDW, and offshore buoy locations. In addition to the meso-eta forecasts, observational data were 
archived daily in order to perform the case study analyses. These data include surface, rawinsonde, 
50-MHz and 915-MHz profiler, and Geostationary Operational Environmental Satellite (GOES-8) 
visible and infrared (IR) satellite data. Other data such as KSC/CCAS mesonet towers, WSR-88D 
radar, and cloud observations from launch reconnaissance aircraft are available to verify these 
phenomenon. 
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The AMU daily weather forecast discussion was held during the warm season evaluation period 
(May through August 1996) on Monday through Friday at approximately 1330 UTC. In part, this 
component of the subjective evaluation was designed to simulate how operational forecasters may 
use the 0300 UTC cycle of meso-eta model to assist in forecasting the onset, depth, and motion of the 
east and/ or west coast sea breeze, the occurrence and severity of convection within 25 miles of XMR, 
and the occurrence of steady state winds in excess of 18 kt at XMR over the subsequent 12 to 24 h! 
These phenomenon are being evaluated due to their primary importance for evaluation of LCC, 
Shuttle FR, and because severe weather associated with thunderstorms can be hazardous to 
equipment and personnel performing ground operations at KSC/CCAS. One member of the AMU 
staff lead the weather briefing and filled out a daily evaluation worksheet. The worksheet was used 
to record the availability of meso-eta data, note the occurrence and movement of east and/or west 
coast sea breezes and thunderstorms, log periods when wind speeds exceed 18 kt, and identify days 
suitable for case studies. The data from the Melbourne WSR-88D radar, geostationary satellites 
(GOES-8), KSC 50-MHz profiler, and KSC/CCAS mesonet towers are available in real time to help 
verify the specific phenomenon of interest. 

The case studies are designed to focus on propagation and intensity of tropical waves, cloud 
ceilings and thickness, propagation of cold fronts, and mesoscale wind features identified from time- 
height sections of 50-MHz or 915-MHz profiler data that are not resolved by 12-h rawinsonde 
observations. The analyses of mesoscale wind features include cases from the warm and cool season 
while the analyses of clouds and cold fronts (tropical waves) are limited to cases from the cool 
(warm) season only. The cases are selected based on the occurrence or existence of tropical waves 
and cold fronts that are within 500-1000 km of KSC/CCAS or for challenging forecasts such as the 
development of stratocumulus clouds that occurs frequently in the cool season following the passage 
of a cold front. The analyses for each case study are limited so that a number of cases can potentially 
be examined to quantify the added value (if any) provided by the 29-km eta model in forecasting the 
aforementioned aspects of tropical waves, clouds, cold fronts, and strong winds. 

3.4 Pre-Final Report Discussions 

At the request of the AMU Chief, AMU personnel conducted informal discussions /briefings with 
SMG, 45WS, and NWS MLB to review results of the meso-eta evaluation prior to outlining and 
writing this final report. Note that these briefings were not part of the original evaluation protocol 
designed by the technical working group. The discussions were designed to (1) provide a status of 
the year-long AMU evaluation of the 29-km eta model, (2) solicit feedback from each group, and (3) 
determine what aspects of the evaluation results are most operationally useful based on prioritization 
by SMG, 45WS, and NWS MLB. These discussions were very informative and every attempt is made 
to incorporate as many of the suggestions and requests in this report which are consistent with the 
original model evaluation protocol. 
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4,0 Objective Evaluation Results 


The results presented in this section focus on the objective verification of meso-eta point forecasts 
for XMR, TBW, and EDW as described in section 3.2. Objective verification of point forecasts 
provides a stringent test of model capabilities. Station observations sample many temporal and 
spatial scales of atmospheric phenomena, some of which cannot be resolved by the model. As a 
result, point verification should benefit higher resolution models which resolve finer scales of motion 
but it does tend to give a more pessimistic view of model performance than gridded verification. 
Nevertheless, point verifications are of more interest to end users of model guidance since, 
ultimately, forecasters want to understand characteristic strengths and weaknesses in model forecasts 
at specific locations. 

Results of surface, upper air, and convective parameter verifications are presented in sections 4.1, 
4.2, and 4.3, respectively. Examination of results for every parameter indicates that error 
characteristics are qualitatively similar under both westerly and easterly wind regimes. For example, 
positive and negative biases appear with similar magnitudes at approximately the same time and 
height. For this reason, all available data are combined and results shown below are not stratified by 
wind regime. Attempts to identify statistically significant differences between sample means at each 
station are not performed because sample sizes are rather small and because such efforts are beyond 
the scope of the current evaluation. Speculations about the source of model errors are provided 
where appropriate but should only be viewed as possibilities which are consistent with the statistical 
results. 

4.1 Surface Forecasts 

Examination of biases, RMS errors, and standard deviations for all the surface parameters 
outlined in section 3.2 indicates that diurnal fluctuations exist in the time series. That is, positive or 
negative biases occur at approximately the same time of day in either the 0300 or 1500 UTC forecast 
cycle. Since statistics from the 1500 UTC cycle provide little additional information, only statistics 
from the 0300 UTC forecast cycles are shown in the following sub-sections. In order to facilitate 
comparison, warm and cool season results for surface parameters are presented side by side in Figs. 

4.1 through 4.11. Forecast consistency of surface parameters exhibits a different behavior and is 
considered below in section 4.1.8. 

4.1.1 2-m Temperature 

Surface 2-m temperature biases at XMR and TBW generally range within ±2 °C during both warm 
and cool seasons (Figs. 4.1a, b). Forecast temperatures at EDW are on average colder than observed, 
with biases reaching about -5 8 C. In conjunction with these larger negative biases at EDW, RMS 
errors reach nearly 6 °C and are also larger than the RMS errors at XMR and TBW (Figs. 4.1c, d). As 
discussed in section 3.2 (Eq. 4), the bias and standard deviation represent systematic and non- 
systematic model errors which both make contributions to the total RMS error. In fact, except for the 
early part of the cool season forecasts, standard deviations at EDW are comparable in magnitude to 
those at XMR and TBW (Figs. 4.1e, f). Therefore, the larger contribution to the total error at EDW is 
from the bias or systematic model error. One possible explanation for this systematic model 
deficiency in forecast temperatures at EDW may be that the forecast point data extracted from the 
model are almost 250 m lower than the actual station elevation. 
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2-m Temperature (°C) 



UTC Hour, Warm Season UTC Hour, Cool Season 


Figure 4.1. Bias, RMS error, and standard deviation of 2-m temperature (*C) for XMR (solid lines), 
TBW (dotted lines), and EDW (dashed lines). Panels a (b), c (d), and e (f) show warm (cool) season 
error statistics plotted as a function of verification time. 
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4.1.2 2-m Dew Point Temperature 

Biases in 2-m dew point temperatures at XMR and TBW during the warm season range generally 
within ±2 # C (Fig. 4.2a). At EDW, however, warm season dew point biases are mostly positive, with 
some values in excess of 4 e C. When viewed in conjunction with the 2-m temperature bias (Fig. 4.1a), 
the net effect is that warm season forecasts at EDW tend to be cool and moist, particularly during the 
first half of the forecast period. Another interesting point is that warm season standard deviations at 
EDW are greater than those at XMR and TBW with values ranging from about 2.5 to 4 °C (Fig. 4.2e). 
Since warm season dew point temperature biases and standard deviations are each relatively large at 
EDW, both systematic and non-systematic model errors make substantial contributions to the total 
RMS error for this location (Fig. 4.2c). 

During the cool season, 2-m dew point temperature biases at all three stations are generally larger 
than warm season values (Fig. 4.2b). In particular, dew point temperature biases at TBW range from 
about -2 to 4 °C while at XMR, a moist bias of 3 to 4 °C is evident throughout much of the forecast 
cycle. Moreover, the gradual increase in bias and RMS error at XMR (Figs. 4.2b, d) indicates a 
decrease in forecast accuracy with time for this location. The difficulties in forecasting cool season 
2-m dew point temperatures at XMR could be caused by excessive onshore surface winds (see section 
4.1.4 below) leading to an overestimation of evaporation within the model's parameterized oceanic 
boundary layer. In fact, the vertical moisture flux from the model's oceanic sublayer is influenced by 
the Reynolds number which is a function of horizontal wind speed (Janjic 1994). One way to 
determine if dew point temperature errors are larger for onshore versus offshore flow would be to 
stratify the statistics by surface wind direction. In accordance with the original evaluation protocol, 
statistics were stratified by the 950- to 600-mb layer-average wind direction but did not reveal any 
substantial regime dependencies at either station. However, it is still possible that 2-m dew point 
temperature forecast errors are closely related to changes in surface wind direction/speed, especially 
at a coastal location such as XMR. 

4.1.3 Mean Sea-Level Pressure 

Biases in mean sea-level pressure (MSLP) at XMR and TBW are within ±1.5 mb throughout both 
seasonal evaluation periods (Figs. 4.3a, b). At EDW however, biases range from about -2 to 3.5 mb. 
In fact, cool season RMS errors at EDW increase with time from about 2 to 3.5 mb which indicates a 
gradual loss in forecast accuracy at that location (Fig. 4.3d). The 250 m elevation difference between 
forecasts and observations could again be one explanation for the MSLP errors encountered at EDW. 

4.1.4 10-m Wind Speed 

Warm season biases in the 10-m wind speed range from 0 to -5 m s' 1 at EDW and from -1 to 2 m 
s' 1 at XMR and TBW (Fig. 4.4a). In comparison, cool season biases range from 0 to -2 m s' 3 at EDW 
and 0 to 3.5 m s' 1 at XMR and TBW (Fig. 4.4b). Therefore, 10-m wind speed forecasts at XMR and 
TBW are generally too fast while those at EDW tend to be slow. Moreover, cool season wind speed 
biases at XMR are about 1 m s' 1 faster than those during the warm season. In fact, increases in the 
bias and standard deviation during the cool season at XMR both combine to produce RMS errors 
which are about 1.5 m s* 1 larger than observed during the warm season (Figs. 4.4c, d). Also of interest 
is the fact that warm season forecast errors at EDW follow a diurnal cycle with the largest values 
occurring between about 1500 and 0300 UTC (Figs. 4.4a, c, e). 
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2-m Dew Point Temperature (°C) 



UTC Hour, Warm Season UTC Hour, Cool Season 


Figure 4.2. Bias, RMS error, and standard deviation of 2-m dew point temperature (*C) for XMR 
(solid lines), TBW (dotted lines), and EDW (dashed lines). Panels a (b), c (d), and e (f) show warm 
(cool) season error statistics plotted as a function of verification time. 


12 










Figure 4.3. Bias, RMS error, and standard deviation of mean sea-level pressure (mb) for XMR (solid 
lines), TBW (dotted lines), and EDW (dashed lines). Panels a (b), c (d), and e (f) show warm (cool) 
season error statistics plotted as a function of verification time. 
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10-m Wind Speed (ms') 



UTC Hour, Warm Season UTC Hour, Cool Season 


Figure 4.4. Bias, RMS error, and standard deviation of 10-m wind speed (m s* 1 ) for XMR (solid lines), 
TBW (dotted lines), and EDW (dashed lines). Panels a (b), c (d), and e (f) show warm (cool) season 
error statistics plotted as a function of verification time. 
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One possible explanation for the increase in cool season wind speed biases at XMR may be a 
combination of model resolution and the fact that XMR is within about 3.5 km of the coast line. As 
winds move from sea to land an internal boundary layer develops quickly with distance from the 
shore (Powell et al. 1996). Enhanced surface friction over land slows observed winds within this 
boundary layer. Although these processes are parameterized in the meso-eta model's planetary 
boundary layer scheme (Janjic 1994), its 29-km grid point resolution is likely not sufficient to resolve 
the rapid decrease of wind speed due to friction within the short distance it is actually observed. The 
impact of this resolution shortcoming should be greater during the cool season with the occurrence of 
strong onshore gradient winds. As with the 2-m dew point temperature bias (section 4.1.2), these 
errors could be further investigated by performing wind regime stratifications based on the 10-m 
wind direction instead of the 950- to 600-mb layer-averaged wind direction specified for this study. 

4.1.5 10-m Wind Direction 

Biases in the 10-m wind direction during both warm and cool seasons exhibit a great deal of 
hour-to-hour fluctuation but are all within ±30° (Figs. 4.5a, b). Negative (positive) biases indicate that 
forecast wind directions are on average more counter-clockwise (clockwise) from the observed wind 
direction. Standard deviations range from about 30 to 90°, indicating that much of the error is due to 
random, non-systematic type variabilities in wind direction (Figs. 4.5e, f). Since the model cannot 
temporally or spatially resolve many local effects which influence wind direction, especially when 
wind speeds are light, the magnitude of variability in wind direction errors is not surprising. 

4.1.6 10-m Wind Speed Persistence 

Because of their importance to operational evaluation of LCC and FR, meso-eta forecasts of 10-m 
wind speed are benchmarked against 1, 2, 3, and 6-h wind speed persistence forecasts. For example, 
3-h persistence uses the current wind speed observation as a forecast for wind speeds at the same 
location three hours into the future. Persistence errors are calculated by subtracting the latest wind 
speed observation from the past observation, or persistence forecast. At XMR, mean errors in wind 
speed persistence forecasts during both seasons exhibit a diurnal cycle which increases in magnitude 
with lead time (Figs. 4.6a, b). Specifically, biases in the 1-h persistence forecasts are about 1.5 m s' 1 
smaller than those of the 6-h forecasts. It is not surprising that persistence forecasts tend to be slow 
(fast) around 1800 (0300) UTC since wind speeds generally increase (decrease) during daytime 
(nighttime) hours. RMS errors in persistence forecasts at XMR are mostly smaller than those of the 
meso-eta forecasts, especially during the cool season (Figs. 4.6c, d). These results suggest that, on 
average, persistence provides greater utility than the meso-eta model for 10-m wind speed forecasts 
at XMR. 

At TBW, a diurnal cycle appears in the wind speed persistence biases (Figs. 4.7a, b) which is 
nearly identical in timing and magnitude to that observed at XMR (Figs. 4.6a, b). Again, note that 
biases in the 1-h persistence forecasts are about 1.5 m s' 1 smaller than those of the 6-h forecasts. On 
the other hand, model forecast biases and RMS errors at TBW (Figs. 4.7a-d) are slightly smaller than 
those at XMR (Figs. 4.6a-d), particularly during the cool season. As a result of this subtle decrease in 
model error at TBW, meso-eta forecasts of 10-m wind speed are occasionally more accurate than 3-h 
and 6-h persistence forecasts. 

Biases in wind speed persistence forecasts at EDW range from ±1 m s _1 at 1 h to ±4 m s* 1 at 6 h 
(Figs. 4.8a, b). However, biases and RMS errors for meso-eta wind speed forecasts are mostly larger 
than those made by persistence (Figs. 4.8a-d). Therefore, persistence usually provides greater 
accuracy than the meso-eta model in 10-m wind speed forecasts at EDW. 
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10-m Wind Direction (°) 



UTC Hour, Warm Season UTC Hour, Cool Season 


Figure 4.5. Bias, RMS error, and standard deviation of 10-m wind direction (') for XMR (solid lines), 
TBW (dotted lines), and EDW (dashed lines). Panels a (b), c (d), and e (f) show warm (cool) season 
error statistics plotted as a function of verification time. 
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Figure 4.6. Bias, RMS error, and standard deviation of 10-m wind speed persistence (m s' 1 ) and 
forecast 10-m wind speed (m s' 1 ) for XMR. Wind speed persistence at 1, 2, 3, and 6-h are shown with 
thin solid, dotted, short-dashed, and long dashed lines, respectively. The thick solid line in each 
panel depicts the corresponding error statistic for 10-m wind speed as shown in Fig. 4.4. Panels a (b), 
c (d), and e (f) show warm (cool) season error statistics plotted as a function of verification time. 
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TBW 10-m Wind Speed Persistence (ms') 
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Figure 4.7. Bias, RMS error, and standard deviation of 10-m wind speed persistence (m s -1 ) and 
forecast 10-m wind speed (m s l ) for TBW. Wind speed persistence at 1, 2, 3, and 6-h are shown with 
thin solid, dotted, short-dashed, and long dashed lines, respectively. The thick solid line in each 
panel depicts the corresponding error statistic for 10-m wind speed as shown in Fig. 4.4. Panels a (b), 
c (d), and e (f) show warm (cool) season error statistics plotted as a function of verification time. 
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Figure 4.8. Bias, RMS error, and standard deviation of 10-m wind speed persistence (m s' 1 ) and 
forecast 10-m wind speed (m s' 1 ) for EDW. Wind speed persistence at 1, 2, 3, and 6-h are shown with 
thin solid, dotted, short-dashed, and long dashed lines, respectively. The thick solid line in each 
panel depicts the corresponding error statistic for 10-m wind speed as shown in Fig. 4.4. Panels a (b), 
c (d), and e (f) show warm (cool) season error statistics plotted as a function of verification time. 
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4.1.7 10-m Wind Direction Persistence 


Biases in both model and persistence forecasts of wind direction at XMR range from about -30 to 
15 but vary widely by hour (Figs. 4.9a, b). In comparison, standard deviations of wind direction 
errors range from about 30 to 90* (Figs. 4.9e, f). These results suggest that substantial variability is 
present in both model and persistence forecasts of wind direction. RMS errors in model forecast 
wind directions are generally comparable in magnitude to those of the 3-h and 6-h persistence 
forecasts (Figs. 4.9c, d). Therefore, the utility provided by meso-eta forecasts of 10-m wind direction 
may be comparable to the use of persistence forecasts with lead times greater than 3 hours. The same 
discussion applies to model and persistence forecasts of 10-m wind direction errors at TBW (Fig 
4.10). 

Biases in both model and persistence forecasts of wind direction at EDW are within about ±30" 
but vary widely by hour (Figs. 4.11a, b). A diurnal cycle is evident as maximum (minimum) errors 
occur around 0900 (1800) UTC. During the warm season, this diurnal cycle is also apparent in both 
the RMS errors and standard deviations as values range widely from about 30 to 90° (Figs. 4.11c, e). 
RMS errors and standard deviations are slightly larger during the cool season, with values ranging 
from about 60 to 90* (Figs. 4.11d, f). Since model forecast errors of wind direction at EDW during 
both seasons lie within the range of errors produced by persistence forecasts, Fig. 4.11 collectively 
demonstrates that model forecasts are comparable in accuracy with 2 to 3-h persistence forecasts. 

4.1.8 Forecast Consistency 

As described in section 3.2 (Eq. 5), forecast consistency describes the average difference between 
subsequent forecasts verifying at the same time but initialized 12-h apart. Again, smaller (larger) 
values of consistency indicate that on average model runs agree more (less) closely with one another. 
Although consistency calculations compare two subsequent forecasts valid at the same time, results 
are presented as a function of forecast duration from 12- to 33-h. This is done because direct forecast 
comparisons are independent of real time and because this method of presentation emphasizes the 
12-h lag between subsequent forecasts. Therefore, graphs shown on the left (right) hand side of Figs. 
4.12 and 4.13 present results determined by subtracting newer 1500 (0300) UTC forecasts from older 
0300 (1500) UTC forecasts from 12-h through 33-h. 

For most parameters, consistency between subsequent model runs are smaller than the 
corresponding biases, especially toward the end of the forecast period (Figs. 4.12 and 4.13). For 
example, 10-m wind speed consistency is smaller than the 10-m wind speed bias shown in Fig. 4.4. 
Also, at forecast hour twelve (12), 2-m dew point temperature consistency during the cool season at 
XMR is near 2 *C (Fig. 4.13d) while the bias in 2-m dew point temperature at XMR reaches nearly 4 *C 
(Fig. 4.2b). These results suggest that model forecasts tend to agree more closely with one another 
than with observations. Several parameters such as 2-m temperature, 2-m dew point temperature, 
and 10-m wind speed exhibit larger values of consistency at the beginning of the verification period 
than towards the end of the period. This result may be explained by reiterating that consistency at 
the start of the verification period measures differences between older 12-h forecasts and newer 0-h 
forecasts updated with observations. At later forecast hours, the observation-based initial conditions 
have less impact on the model's depiction of the atmosphere. Therefore, certain parameters tend to 
become more consistent with time as the forecasts respond less to forcing by the initial conditions. 


20 



XMR 10-m Wind Direction Persistence (°) 
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UTC Hour, Cool Season 


Figure 4.9. Bias, RMS error, and standard deviation of 10-m wind direction persistence (°) and 
forecast 10-m wind direction (°) for XMR. Wind direction persistence at 1, 2, 3, and 6-h are shown 
with thin solid, dotted, short-dashed, and long dashed lines, respectively. The thick solid line in each 
panel depicts the corresponding error statistic for 10-m wind direction as shown in Fig. 4.5. Panels a 
(b), c (d), and e (f) show warm (cool) season error statistics plotted as a function of verification time. 
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Figure 4.10. Bias, RMS error, and standard deviation of 10-m wind direction persistence (') and 
forecast 10-m wind direction (*) for TBW. Wind direction persistence at 1, 2, 3, and 6-h are shown 
with thin solid, dotted, short-dashed, and long dashed lines, respectively. The thick solid line in each 
panel depicts the corresponding error statistic for 10-m wind direction as shown in Fig. 4.5. Panels a 
(b), c (d), and e (f) show warm (cool) season error statistics plotted as a function of verification time. 
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Figure 4.11. Bias, RMS error, and standard deviation of 10-m wind direction persistence ( # ) and 
forecast 10-m wind direction (°) for EDW. Wind direction persistence at 1, 2, 3, and 6-h are shown 
with thin solid, dotted, short-dashed, and long dashed lines, respectively. The thick solid line in each 
panel depicts the corresponding error statistic for 10-m wind direction as shown in Fig. 4.5. Panels a 
(b), c (d), and e (f) show warm (cool) season error statistics plotted as a function of verification time. 
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Figure 4.12. Warm season forecast consistency of temperature (*C), dew point temperature (*C), mean 
sea-level pressure (mb), wind speed (m s' 1 ), and wind direction (’) at XMR (solid lines), TBW (dotted 
lines), and EDW (dashed lines). Panels a (b), c (d) , e (f) , g (h) , and i (j) show consistency calculated 
by subtracting newer 1500 (0300) UTC forecasts from older 0300 (1500) UTC forecasts plotted as a 
function of forecast duration. Consistency plots begin at forecast hour twelve (12) because 
subsequent forecasts initialized 12-h apart first overlap at that time. 
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Figure 4.13. Cool season forecast consistency of temperature (*C), dew point temperature ( # C), mean 
sea-level pressure (mb), wind speed (m s' 1 ), and wind direction (*) at XMR (solid lines), TBW (dotted 
lines), and EDW (dashed lines). Panels a (b), c (d) , e (f) , g (h) , and i (j) show consistency calculated 
by subtracting newer 1500 (0300) UTC forecasts from older 0300 (1500) UTC forecasts plotted as a 
function of forecast duration. Consistency plots begin at forecast hour twelve (12) because 
subsequent forecasts initialized 12-h apart first overlap at that time. 
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4.2 Upper Air Forecasts 

Examination of all statistics for the upper air parameters outlined in section 3.2 does not reveal 
any substantial differences between the 0300 and 1500 UTC forecast cycles. Moreover, the statistics 
do not reveal any obvious differences when the results are stratified by the 950- to 600-mb layer- 
averaged wind direction. For example, biases generally occur with comparable magnitudes at about 
the same height regardless of forecast cycle or wind regime classification. In light of these findings, 
the bias, RMS error, and standard deviation at XMR, TBW, and EDW are presented by season as a 
function of pressure and verification period. In addition to the graphical results, tables are presented 
in the Appendix which list biases and RMS errors in u and v wind components, wind speed, and 
wind direction as a function of height and verification period for XMR and EDW. These statistics are 
listed in tabular format for general interest and applications to Shuttle landing flight rules. 

Since each verification period defined in section 3.2 corresponds to forecasts of greater duration, 
direct comparison of RMS errors from each period enables the identification of possible error growth 
with time. However, examination of results presented below indicates that steady or substantial 
increases in error with time only occur in a few instances. 

4.2.1 Geopotential Height 

To diagnose errors in geopotential height forecasts, GEMPAK was used to extract or calculate all 
upper air parameters from both forecast and observed sounding data at their reported levels. These 
parameters are then input into a program which log-linearly interpolates the data to specified 
pressure and height levels before computing statistics as described in section 3.2. After the statistics 
were computed, vertical profiles of geopotential height error were noisy because observed heights at 
mandatory levels are reported and stored directly in GEMPAK sounding files whereas heights at 
other significant levels are computed by GEMPAK using reported values of temperature and dew 
point temperature. This discrepancy could be corrected by first interpolating all forecast and 
observed temperatures and dew point temperatures to the desired verification levels. Then moist 
hydrostatic heights can be derived using the same method at all levels prior to computing the bias, 
RMS error, and standard deviation. Because the problem was discovered late in the evaluation 
period, there was not sufficient time to update the software and recompute the statistics. Therefore, 
geopotential height errors are not presented in this report. Since derived heights are closely tied to 
virtual temperature, larger height errors should be expected at those levels where large temperature 
and moisture errors are reported. 

4.2.2 Temperature 

Warm season temperature forecasts at EDW are nearly unbiased except above the tropopause 
and at levels below 850 mb (Fig. 4.14c). At XMR and TBW, biases below 700 mb are about 1 *C colder 
than observed whereas above 600 mb they are nearly 2 °C warmer than observed (Figs. 4.14a, b). The 
net effect for warm season forecasts at these Florida stations is a tendency towards a thermally stable 
model atmosphere. Below the tropopause, standard deviations (Figs 4.14g, h, i) indicate that random, 
non-systematic errors in the forecasts are about 1 “C. In comparison, typical RMS uncertainty in 
rawinsonde temperature observations is about 0.6 °C (Hoehne 1980; Ahnert 1991) which suggests that 
more than half of the non-systematic error may be due to measurement uncertainty. 

During the cool season, temperature forecasts at EDW exhibit a cool bias below 700 mb of 1 to 2 
C (Fig. 4.15c). At XMR and TBW, temperature errors above 600 mb and below 850 mb average about 
1 *C warmer than observations (Fig. 4.15a, b). In between, there is a sharp cold bias of nearly -2 *C 
around the 750 mb level. Examination of forecast and observed soundings at XMR throughout the 
cool season (not shown) reveals that this cold bias appears primarily because model forecasts of the 
lower tropospheric inversion are frequently stronger and higher than actually observed. 
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Warm Season Temperature Errors (°C) 



Bias RMS Error Std. Dev. 


Figure 4.14. Warm season bias, RMS error, and standard deviation of temperature (°C) plotted as a 
function of pressure level for XMR (panels a, d, g), TBW (panels b, e, h), and EDW (panels c, f, i). The 
solid, dotted, and dashed lines show the corresponding error statistics for verification periods 1, 2, 
and 3, respectively. The verification periods are defined in Fig. 3.1. 
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Figure 4.15. Cool season bias, RMS error, and standard deviation of temperature ("C) plotted as a 
function of pressure level for XMR (panels a, d, g), TBW (panels b, e, h), and EDW (panels c, f, i). The 
solid, dotted, and dashed lines show the corresponding error statistics for verification periods 1, 2, 
and 3, respectively. The verification periods are defined in Fig. 3.1. 
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4.2.3 Mixing Ratio 


Warm season mixing ratio biases at XMR and TBW (Figs. 4.16a, b) indicate that meso-eta 
forecasts are generally too dry in the lower troposphere, most notably at XMR. Conversely, biases 
above 500 mb suggest that forecasts at these locations tend to retain larger amounts of moisture than 
observed. In combination with the cool lower tropospheric temperature biases discussed in the 
previous section, these results suggest that warm season model forecasts at XMR and TBW are 
typically more stable than observed. Section 4.3 will demonstrate that stability index errors also 
indicate a stable bias. At EDW, middle tropospheric mixing ratio biases are near zero during the 
warm season (Fig. 4.16c). However, upper and lower tropospheric forecasts at EDW tend to retain 
larger amounts of moisture than observed. During the cool season, mixing ratio biases at all three 
stations are slightly wet or near zero at most levels (Figs. 4.17a, b, c) except for the dry layer which 
appears between 950 and 800 mb at XMR and TBW. 

RMS errors for both warm and cool seasons (Figs. 4.16-4.17d, e, f) drop from around 2 g kg' 1 at 
low-levels (1.5 g kg 1 at EDW) to near zero at 200 mb, where there is very little water vapor present in 
the atmosphere. At XMR and TBW in particular, it is interesting to note that warm season RMS 
errors in the lower troposphere increase slightly with verification period, thereby indicating a growth 
in forecast error with time (Figs. 4.17d, e). Results shown in Figs. 4.16-4.17d-f are consistent with 
those of Rogers et. al (1996), who show 24-h RMS errors in specific humidity from 48-km eta model 
forecasts across the United States during September 1994 ranging from nearly 2 g kg' 1 at 1000 mb to 
less than 0.1 g kg' 1 at 250 mb (see their Fig. 7). Note that these calculations for mixing ratio errors are 
not normalized by magnitude and are therefore not representative of percent errors as the mixing 
ratio tends toward zero in the upper troposphere. 

4.2.4 Wind Speed 

Warm season wind speed biases below the tropopause at XMR and TBW are less than ±1 m s' 1 
(Figs. 4.18a, b). Forecast wind speeds at EDW are about 2 m s' 1 slower than observed, especially in 
the lower troposphere (Fig. 4.18c). Below the tropopause, RMS errors at XMR and TBW slightly 
exceed 2 m s' 1 while at EDW they reach nearly 4.5 m s' 1 (Figs. 4.18d, e, f). Moreover, RMS errors in 
wind speed forecasts at EDW appear to increase with verification period by about 1 m s' 1 which 
indicates error growth with time. Since biases are small at all three stations, these results suggest that 
the model suffers little systematic error in wind speeds. In fact, relatively large standard deviations 
of 2 to 5 m s 1 (Figs. 4.18g, h, i) reveal that random variations in forecast and/or observed wind 
speeds account for a large portion of the total wind speed error. For comparison, uncertainties in 
rawinsonde wind speed measurements are about 3.1 m s' 1 (Hoehne 1980), which suggests that a large 
component of the total wind speed error may result from observational uncertainties in wind speed. 

During the cool season, wind speed biases at XMR and TBW are less than ±1.5 m s* 1 (Figs. 4.19a, 
b). At EDW, wind speed biases exhibit strong variations with height but remain within ±2 m s' 1 (Fig. 
4.19c). RMS errors at XMR and TBW range from 2 m s* 1 in the lower troposphere to just over 4 m s' 1 
in the upper troposphere while at EDW, errors reach nearly 6 m s' 1 (Figs. 4.19d, e, f). As in the warm 
season, the increase in middle tropospheric RMS errors with verification period at EDW suggests that 
forecast errors grow with time for this location. The relatively large standard deviations of 2 to 6 m 
s' 1 (Figs. 4.19g, h, i) also indicate that as in the warm season, non-systematic variations in forecast 
and/or observed wind speeds account for a large portion of the total wind speed error. 
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Figure 4.16. Warm season bias, RMS error, and standard deviation of mixing ratio (gm kg' 1 ) plotted as 
a function of pressure level for XMR (panels a, d, g), TBW (panels b, e, h), and EDW (panels c, f, i). 
The solid, dotted, and dashed lines show the corresponding error statistics for verification periods 1, 
2, and 3, respectively. The verification periods are defined in Fig. 3.1. 
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Figure 4.17. Cool season bias, RMS error, and standard deviation of mixing ratio (gm kg' 1 ) plotted as 
a function of pressure level for XMR (panels a, d, g), TBW (panels b, e, h), and EDW (panels c, f, i). 
The solid, dotted, and dashed lines show the corresponding error statistics for verification periods 1, 
2, and 3, respectively. The verification periods are defined in Fig. 3.1. 
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Figure 4.18, Warm season bias, RMS error, and standard deviation of wind speed (m s" 1 ) plotted as a 
function of pressure level for XMR (panels a, d, g), TBW (panels b, e, h), and EDW (panels c, f, i). The 
solid, dotted, and dashed lines show the corresponding error statistics for verification periods 1, 2, 
and 3, respectively. The verification periods are defined in Fig. 3.1. 


32 


EDW TBW XMR 






X 

6 


e 

p 

CO 

CO 

<D 

Uh 





<D 

U-i 

P 

CO 

CO 

<L> 


(X. 


X 

S 


<D 

Vh 

P 

CO 

CO 

p 




Cool Season Wind Speed Errors (m s ! ) 





-4 -2 0240246802468 


Bias RMS Error Std. Dev. 


Figure 4.19. Cool season bias, RMS error, and standard deviation of wind speed (m s* 1 ) plotted as a 
function of pressure level for XMR (panels a, d, g), TBW (panels b, e, h), and EDW (panels c, f, i). The 
solid, dotted, and dashed lines show the corresponding error statistics for verification periods 1, 2, 
and 3, respectively. The verification periods are defined in Fig. 3.1. 
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4.2.5 Wind Direction 


Warm season wind direction forecasts at XMR and TBW are nearly unbiased, as mean errors lie 
within ±10* (Figs. 4.20a, b). Biases at EDW are within ±20" and demonstrate greater variability with 
height, especially within the lower troposphere (Fig. 4.20c). Moreover, warm season wind direction 
forecasts at EDW are typically negative, or counter-clockwise relative to observed values except for 
the 750 to 850 mb layer where biases for verification periods 2 and 3 are positive. Although forecast 
wind direction biases are reasonably small, RMS errors and standard deviations approach 60’ and are 
nearly twice as large at XMR and TBW as they are at EDW (Figs. 4.20d-i). Hoehne (1980) reports that 
observational uncertainty in wind direction decreases with increasing wind speeds from 14* at 10 kt 
to 2* at 120 kt. In addition to the fact that observational uncertainty for wind direction is a function of 
wind speed, the accuracy of forecast wind directions may also be influenced by wind speed. At most 
levels, average warm season observed wind speeds (not shown) at EDW are about 5 to 15 kt faster 
than those at XMR and TBW with larger differences occurring in the upper troposphere. Average 
warm season forecast wind speeds exhibit the same characteristics. Since mean forecast and 
observed wind speeds are faster at EDW, it is not surprising that standard deviations in wind 
direction are smaller at EDW than those at XMR and TBW. That is, the difference in wind direction 
standard deviations between EDW and XMR and TBW could be accounted for, in part, by difficulties 
in both observing and forecasting wind directions accurately at low wind speeds. 

Cool season errors in wind direction forecasts are similar to those of the warm season with a few 
exceptions (Fig. 4.21). First, while there is no longer a lower tropospheric positive bias in wind 
direction forecasts at EDW, a positive bias does appear for lower tropospheric wind directions at 
TBW (Figs. 4.21b, c). Second, RMS errors and standard deviations at XMR and TBW decrease relative 
to warm season values above 700 mb (Figs. 4.21d, e, g, h). During the cool season, both forecast and 
observed average wind speeds (not shown) increase relative to warm season speeds by about 15 to 25 
kt at XMR and TBW. Therefore, the upper tropospheric decrease in RMS errors and standard 
deviations in wind direction at XMR and TBW could again be explained by the idea that both forecast 
and observed wind directions are more accurate at higher wind speeds.- 

4.2.6 Consistency 

Model forecast consistency for upper air data is calculated as a function of pressure and time (Eq. 
5). As with the surface consistency calculations discussed in section 4.1.8, upper air calculations 
begin 12-h into the older forecast and proceed through the end of 33-h forecast period. However, 
since upper air calculations have the additional pressure-level dimension, only consistency values at 
12, 21, and 30 h are presented for brevity. These times are selected because they represent values at 
even intervals which correspond to the beginning, middle, and end of the period over which 
consistency is calculated. Moreover, these times match reasonably well with each of the three periods 
defined in section 3.2 for verification of upper air forecasts with observations. Because there are no 
substantial diurnal signatures in the upper air consistency values, the results plotted in Figs. 4.22 and 
4.23 make use of all available forecasts from both the 0300 UTC and 1500 UTC forecast cycles. 

Examination of Figs. 4.22 and 4.23 reveals that upper air consistency values are small compared 
to the corresponding forecast biases shown previously in Figs. 4.14-4.21. That is, the average 
difference between subsequent forecasts is small relative to the mean forecast error. Larger 
differences are commonly found at 12 h (solid line in Figs. 4.22 and 4.23), which measures the 
consistency between the current 0-h initialization/ forecast and the 12-h forecast from the previous 
model run. The fact that larger differences are found for upper air consistency more often at 12 h 
rather than at 21 h or 30 h is not surprising because newer forecasts are updated with observations at 
12 h, as discussed for surface consistency calculations in section 4.1.8. 
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Figure 4.20. Warm season bias, RMS error, and standard deviation of wind direction (°) plotted as a 
function of pressure level for XMR (panels a, d, g), TBW (panels b, e, h), and EDW (panels c, f, i). The 
solid, dotted, and dashed lines show the corresponding error statistics for verification periods 1, 2, 
and 3, respectively. The verification periods are defined in Fig. 3.1. 
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Figure 4.21. Cool season bins, RMS error, snd stdndsrd deviation of wind direction (°) plotted as a 
function of pressure level for XMR (panels a, d, g), TBW (panels b, e, h), and EDW (panels c, f, i). The 
solid, dotted, and dashed lines show the corresponding error statistics for verification periods 1, 2, 
and 3, respectively. The verification periods are defined in Fig. 3.1. 
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Figure 4.22. Warm season forecast consistency of temperature (°C), mixing ratio (gm kg’ 1 ), wind 
speed (m s’ 1 ), and wind direction (*) plotted as a function of pressure at XMR (panels a, d, g, j), TBW 
(panels b, e, h, k), and EDW (panels c, f, i, 1). The solid, dotted, and dashed lines show forecast 
consistency at 12-h, 21-h, and 30-h, respectively. See text for details of verification times. 
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Figure 4.23. Cool season forecast consistency of temperature fC), mixing ratio (gm kg '), wind speed 
(m s' 1 ), and wind direction (“) plotted as a function of pressure at XMR (panels a, d, g, j), TBW (panels 
t*/ fi/ h/ k), and EDW (panels c, f, i, 1). The solid, dotted, and dashed lines show forecast consistency at 
12-h, 21-h, and 30-h, respectively. See text for details of verification times. 
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Many characteristic forecast biases identified in Figs. 4.14-4.21 appear, particularly with the 12-h 
consistency values, in the consistency profiles shown in Figs. 4.22 and 4.23. For example, the 
cool/ dry nature of lower tropospheric warm season forecasts at XMR and TBW is discernible in the 
temperature and mixing ratio consistency profiles shown in Figs. 4.22a, b, d, and e. Similarly, the 
model's tendency to forecast cool season lower tropospheric temperature inversions which are higher 
and stronger than observed at XMR and TBW is evident between 600 and 800 mb (Figs. 4.23a, b). 
Wind speeds exhibit larger inconsistencies in the upper troposphere during both seasons, while wind 
direction changes are within ±10°. Overall, subsequent forecasts of upper air parameters tend to be 
more consistent with each other than with observations. Moreover, the differences between 
subsequent forecasts are in agreement with the forecast biases identified in sections 4.2.2 through 
4.2.5. 

4.3 Convective Indices 

The bias and RMS error for warm and cool season convective indices described in section 3.2 
(Table 3.1) are presented in Table 4.1 for XMR and TBW as a function of verification period. Results 
for EDW are not shown because convection is usually not a concern at that location. The convective 
parameters shown in Table 4.1 are typically used to help forecast the probability of thunderstorm 
occurrence on a given day. Convective activity is almost certain to occur, for example, when the 
KINX approaches 40. When used alone, the errors in forecast convective parameters such as KINX on 
a particular day may actually be large enough to provide misleading information regarding the 
likelihood for thunderstorm development. 

Statistical calculations were performed using all available data from both the 0300 UTC and 1500 
UTC model runs and corresponding observations. Although calculations were originally stratified 
by forecast cycle and 950- to 600-mb layer-averaged wind regime, only the combined results are 
shown since no substantial differences were identified in the stratified statistics. Standard deviations 
are omitted because in most cases they are similar in magnitude to the RMS errors and because they 
can be calculated directly using Eq. 4. 

Table 4.1. Bias and RMS error for warm and cool season convective parameters at XMR 
and TBW. See Fig. 3.1 and Table 3.1 for definitions of verification periods and convective 
parameter acronyms, respectively. 



39 






During the warm season, negative biases in PWAT and CAPE and positive biases in LIFT suggest 
that forecast soundings are typically drier and more stable than observed (Table 4.1). In particular, 
CAPE forecasts at XMR are nearly 1000 J kg' 1 smaller than observed values and RMS errors exceed 
1200 J kg 1 . These errors are consistent with upper air warm season biases in mixing ratio and 
temperature at XMR which indicate that forecasts tend to be thermodynamically more stable than 
observed (Figs. 4.14 and 4.16). Later in section 5, it will be demonstrated that the meso-eta model 
typically generates warm season precipitation over Florida too early in the day. One possible 
explanation for this discrepancy between overly stable forecast soundings and excessive precipitation 
is that the model erodes the stable surface layer and breaks the capping inversion more quickly and 
vigorously than observed. Unfortunately, rawinsonde data are not consistently available at sufficient 
temporal resolution throughout the daytime hours to validate this hypothesis. 

Cool season biases and RMS errors in PWAT and CAPE are mostly smaller than the 
corresponding warm season errors. Although the meso-eta model may actually be forecasting these 
indices more accurately during the cool season, another possible reason for the decrease in error may 
result from the fact that forecast and observed values of PWAT and CAPE are generally smaller 
during the cool season. Positive biases in CAPE and KINX and negative biases in LIFT suggest that 
cool season forecasts are more unstable than observed (Table 4.1). These results are consistent with 
the warm and moist unstable cool season temperature and mixing ratio biases shown at low-levels in 
Figs. 4.15 and 4.17. Cool season RMS errors for KINX increase relative to warm season errors. KINX 
measures instability based on the 850- to 500-mb lapse rate, the 850-mb dew point temperature, and 
the 700-mb dew point depression. Examination of biases for each of these parameters reveals that the 
700-mb cold bias in temperature (Figs. 4.15a, b) makes the strongest contribution to the cool season 
KINX errors. 

Positive biases in MDPI during both warm and cool seasons indicate that on average the model 
tends to overforecast the potential for microburst development at XMR and TBW based on 
environmental lapse rates of equivalent potential temperature (Wheeler and Roeder 1996). Biases in 
HLCY are also slightly positive during both seasons, except for verification period 1 at TBW. While 
HLCY biases are small, RMS errors during the cool season are nearly double those for the warm 
season. Since HLCY biases are small and contribute little to the total RMS error, much of the total 
RMS error develops in response to non-systematic variabilities in forecasts and/or observations of 
HLCY (Eq. 4, section 3.2). Since HLCY is a measure of the vertically-integrated wind shear over the 
lowest 3 km of the atmosphere, this variability is consistent with the large standard deviations 
previously noted in section 4.2.5 with regard to wind direction errors (Figs. 4.20, 4.21). Increases in 
RMS errors for HLCY from the warm to cool season may occur because mean cool season forecast 
and observed wind speeds and corresponding wind shear are larger than those during the warm 
season. 

4.4 850 to 500-mb Layer-Averages 

The bias and RMS error for warm and cool season 850 to 500-mb layer-averaged winds and 
relative humidity are presented for XMR, TBW, and EDW in Table 4.2 as a function of verification 
period. In addition to the layer-averaged wind speed and direction errors, statistics are included for 
the ii and v wind components for application to LCC and Shuttle FR. As with convective parameters, 
calculations were performed using all available data from both the 0300 and 1500 UTC model runs 
and corresponding observations. Although calculations were originally stratified by forecast cycle 
and 950- to 600-mb layer-averaged wind regime, only the combined results are shown since no 
substantial differences were identified in the stratified statistics. Standard deviations are omitted 
because in most cases they are similar in magnitude to the RMS errors and because they can be 
directly calculated using Eq. 4. 



Table 4.2. Warm and cool season bias and RMS error for 850- to 500-mb layer-averaged 
u~, and u-wind components, wind speed and wind direction. See Fig. 3.1 for definitions of 
verification periods. 





U-wind (m 

s ‘ l ) 

V-wind (m s* 1 ) 

Wind Speed ( 

m s' 1 ) 

Wind Direction (*) 1 






XMR 

TBW 

EDW 

XMR 

TBW 

EDW 

XMR 

TBW 

EDW 

XMR 

TBW 

EDW 

Warm 

1 

-l 

3 

2 

-1.3 

-0.8 

-0.8 

-0.3 

-0.1 

0.7 

0.0 

0.3 

-0.7 

-5 

-2 

-9 

Season 

2 

-3 

2 

0 

-0.1 

-0.7 

-0.3 

0.0 

-0.4 

-0.3 

0.0 

0.2 

-1.0 

-4 

-7 

-3 

Bias 

3 

-2 

4 

0 

-0.3 

-0.9 

-0.9 

0.1 

-0.1 

0.2 

0.2 

0.4 

-1.3 

-5 

-8 

-9 

Cool 

1 

2 

1 

2 

-1.0 

-1.0 

-0.9 

0.2 

0.0 

0.7 

-0.7 

-0.2 

-0.3 

-3 

-2 

-4 

Season 

2 

2 

2 

0 

-0.6 

-0.6 

-1.6 

0.1 

-0.1 

0.5 

-0.4 

-0.2 

-0.6 

-2 

-2 

-8 

Bias 

3 

1 

1 

1 

-0.6 

-0.7 

-2.4 

0.0 

-0.2 

0.8 

-0.2 

0.1 

-1.1 

-2 

-4 

-10 

Warm 

1 

10 

11 

6 

2.1 

1.7 

2.0 

1.6 

1.3 

1.7 

1.9 

1.8 

1.7 

35 

31 

17 

Season 

2 

12 

13 

6 

1.7 

1.7 

2.4 

1.8 

1.6 

2.2 

1.8 

1.7 

2.1 

34 

36 

33 , 

RMS Error 

3 

13 

13 

6 

1.9 

2.3 

2.8 

2.1 

2.0 

2.2 

2.1 

2.0 

2.8 

39 

47 

28 

Cool 

1 

11 

10 

8 

1.5 

1.6 

1.9 

1.4 

1.3 

2.1 

1.5 

1.7 

1.7 

20 

22 

12 

Season 

2 

12 

10 

9 

1.6 

1.9 

2.5 

1.6 

1.5 

2.5 

1.7 

1.7 

2.3 

21 

22 

30 

RMS Error 

3 

14 

12 

9 

1.9 

2.1 

3.2 

1.9 

2.0 

2.6 

2.1 

2.3 

2.7 

26 

31 

40 


fn general, the results shown in Table 4.2 are consistent with the discussion and figures presented 
in section 4.2. Biases in relative humidity (RH) at all three stations are near zero during both seasons 
with corresponding RMS errors of 5-10% at EDW and 10-15% at XMR and TBW. Observational 
uncertainty in rawinsonde observations of RH are about 2.5% (Ahnert 1991) which may provide a 
strong contribution to the total error in RH forecasts. Because RH varies non-linearly with the 
saturation mixing ratio, direct comparisons cannot be made with the mixing ratio errors shown in 
Figs. 4.16 and 4.17. However, it is interesting to note that the sign of RH biases shown in Table 4.2 
generally correspond well with the sign of mixing ratio biases in the levels between 850 to 500 mb 
(Figs. 4.16 and 4.17). 

RMS errors for u and v wind component forecasts at all three stations range from about 1.5 to 3.2 
m s' 1 during both warm and cool seasons. Negative biases in the u-component winds indicate that 
forecasts are typically slower than observations while biases in the ^-component winds are closer to 
zero. Since u and v wind components make up the total wind speed, it is not surprising that RMS 
errors in wind speed also range from about 1.5 to 2.8 m s* 1 . These results for wind speed errors (Table 
4.2) are consistent with the statistics at individual levels from 850 to 500 mb (Figs. 4.18 and 4.19). 
Biases in wind direction range from -2 to -10°, thereby indicating that forecast wind directions in the 
850 to 500-mb layer tend to be counterclockwise relative to observed directions. RMS errors in wind 
direction are largest during the warm season with values ranging from 17 to 47°. The smaller cool 
season RMS errors range from 12 to 40°. The small biases and relatively larger RMS errors in 850- to 
500-mb layer-averaged wind direction forecasts are consistent with results discussed in section 4.2.5 
and shown in Figs. 4.20 and 4.21. Particularly, the large RMS errors are indicative of substantial 
variability in the forecast and/or observed wind directions. Moreover, the decrease in cool season 
RMS errors relative to warm season values are consistent with the fact that measurement uncertainty 
decreases with increasing wind speeds. 
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5.0 Results of Subjective Evaluation 


The results presented in this section focus on the subjective verification of sea breezes, 
thunderstorms, tropical waves, cold fronts, winds, and clouds. The subjective component of the 
overall meso-eta model evaluation is designed to assess the added value of the meso-eta model in 
forecasting selected aspects of these phenomenon. The evaluation strategy for sea breezes and 
thunderstorms consists of limited case studies of selected events and a seasonal verification on all 
days during the warm season when both observational and forecast data are available. A similar 
procedure is followed for the verification of cold fronts using data collected during the cool season. 
The verification of tropical waves during the warm season is limited by the availability of observed 
cases which were present within the subset domain of gridded forecast data. Finally, case studies of 
interesting wind and cloud forecasts are not performed for reasons discussed in sections 5.5 and 5.6, 
respectively. 

5.1 Sea-Breeze Verification 

The sea/ land breeze is a well-documented mesoscale circulation that affects many coastal areas 
of the world including the peninsula of Florida (Pielke and Segal 1986). Numerous observational 
(e.g. Blanchard and Lopez 1985) and numerical (e.g. Pielke 1974; Boybeyi and Raman 1992) studies 
have demonstrated how the timing and location of convection across Florida is often modulated by 
local interactions between the sea-breeze circulation and prevailing synoptic-scale wind. With its 29- 
km grid point spacing, the horizontal resolution of the meso-eta model is too coarse to resolve the 
detailed, mesoscale structure of both the east and west coast sea breezes that are common along the 
Florida coastlines during the warm season. Nevertheless, the model can forecast certain 
characteristics of the observed sea breeze including a thermally direct circulation that results from 
differential low-level heating across the land/sea interface. The verification consists of both a case 
study and statistics that show how reliably the model can forecast the occurrence of the sea breeze 
over Florida during the warm season from May through August 1996. 

5.1.1 Case Example 

The example presented here is an analysis of forecast and observed sea-breeze development over 
the Florida peninsula on 7 June 1996. This case is chosen because it illustrates the typical 29-km eta 
model signature of the sea breeze that is often forecast during the warm season. Another reason for 
selecting 7 June is the availability of both model and observational data needed for the analysis. The 
following sections present observations of the sea breeze followed by an examination of the meso-eta 
forecast to distinguish clearly the model capabilities and limitations in depicting the structure and 
evolution of the sea-breeze circulation. 

5.1.1.1 Observations 

A limited synoptic overview of the large-scale conditions at 1200 UTC 7 June 1996 is shown in 
Fig. 5.1. This analysis is obtained from the 0-h forecast of the 48-km eta model run initialized at 1200 
UTC 7 June 1996. At that time, the Florida peninsula is under the influence of a ridge axis oriented 
west-to-east across the central portion of the state (Fig. 5.1) and a closed high in the Gulf of Mexico, as 
shown by the 850 mb geopotential heights in Fig. 5.1. The 950- to 650-mb layer-averaged wind 
vectors reveal that lower tropospheric winds are light (< 5 kt) and variable over Florida in association 
with weak geopotential height gradients. Surface winds (not shown) along the east and west coasts 
of Florida are also light and variable with offshore components along sections of both coasts. 
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A peninsula-scale perspective of the sea breeze along Florida's east and west coasts is shown in 
the 4-km GOES visible imagery (Fig. 5.2). The development and movement of the sea breeze is 
inferred from the location of narrow bands of shallow cumulus clouds. It is important to point out 
that sea-breeze circulations are likely present before their signatures appear as bands of visible clouds 
(Weckwerth et al. 1997). However, this limitation is not critical for the case presented here since the 
visible imagery is used to illustrate the qualitative features of the sea breeze rather than to estimate its 
onset time and/ or propagation speed. 


5 . kt -> 



Figure 5.1. Synoptic overview from 0-h forecast of the 48-km eta model run initialized at 1200 UTC 7 
June 1996. The 850 mb geopotential heights (m) contoured every 5 m are shown by thick solid lines. 
Vectors represent the 950- to 650-mb layer-averaged winds. The vector length is proportional to wind 
speed with a representative 5-kt vector shown in the upper right corner of the figure. The larger and 
smaller subset boxes outline the area shown in Figs. 5.2 and 5.3, respectively. The station locations 
for Orlando International Airport (MCO) and buoy 41009 are given by the "M" and "9" symbols, 
respectively. 
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At 1600 UTC 7 June (Fig. 5.2a), a distinct line of clouds associated with the east coast sea breeze 
extends from Jacksonville (JAX) down to the southern tip of the state. A similar line of clouds 
associated with the west coast sea breeze extends northwest from Tampa (TBW) to the Big Bend area 
just south of Tallahassee (TLH). By 2000 UTC 7 June (Fig. 5.2b), the east and west coasts sea-breeze 
circulations propagate toward the center of the state. The cloud band associated with the west coast 
sea breeze along the southwest coast of Florida at 2000 UTC is not clearly evident 4 h earlier at 1600 
UTC (compare Figs. 5.2a, b). As is common during the warm season afternoons in Florida, significant 
deep convection is occurring along sections of the sea breeze. 

A meso-beta scale analysis of the sea breeze during 7 June in east-central Florida is shown by the 
streamlines of gridded wind observations from the KSC/CCAS tower network (Fig. 5.3). The 54-ft 
(16.5-m) level u- and n-component of the wind from the KSC/CCAS network at 5-minute intervals 
are analyzed to a 4-km grid using a two-pass Barnes (1964) scheme. From representative 4-km grid 
intervals shown in Fig. 5.3, it can be inferred that most of the KSC/CCAS tower network is contained 
within a single 29-km eta model grid box. Therefore, the meso-eta model is not expected to resolve 
any of the spatial variability in wind speed or direction that appears in Fig. 5.3. However, the 
observed wind tower data are useful to illustrate the movement of the sea-breeze front across 
KSC/CCAS (e.g. Laird et al. 1995). 



Figure 5.2. The 4-km GOES visible imagery for (a) 1600 UTC 7 June 1996 and (b) 2000 UTC 7 June 
1996. . Station identifiers for Jacksonville, FL (JAX), Tampa Bay, FL (TBW) and Tallahassee, FL (TLH) 
are shown in panel (a). 


The streamlines and 54-ft tower observations plotted in Fig. 5.3a show a light northwest or 
offshore flow at 1200 UTC 7 June. By 1400 UTC 7 June, the sea-breeze front moves to the west of 
KSC/CCAS (beyond the left edge of Fig. 5.3) as indicated by the northeast winds. In addition to the 
wind shift, coastal towers are indicating an increase in wind speed relative to those observed 2-h 
earlier. The tower data plotted in Fig. 5.3 demonstrate that the observed sea breeze forms and passes 
to the west of KSC/CCAS in less than two hours. 
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It is possible to estimate the onset time for the sea breeze across east-central Florida on 7 June by 
examining a time series of wind direction at 5-minute intervals averaged over all non-missing points 
in the 4-km analysis grid. As shown by the solid line in Fig. 5.4, the average wind direction veers 
markedly from about 330 to 60° between 1200 and 1300 UTC in association with the passage of the sea 
breeze across the KSC/CCAS region. Despite the fact that the observed time series shown in Fig. 5.4 
represents grid-averaged wind directions from objectively analyzed tower data at 5-minute intervals, 
there is still a great deal of variability present. In comparison, the 29-km eta model forecast time 
series of wind direction at XMR shown by the dashed line in Fig. 5.4 does not contain the high 
frequency fluctuations that are present in the observations. These results underscore the point made 
in section 4, namely that verifying model forecasts at individual points (or stations) is a very stringent 
test of model capability since the observations contain scales of motion not resolved by the model. 
The details from the 29-km eta model forecast of the sea breeze for this case including the point 
forecast of wind direction are discussed in section 5.1. 1.2. 



Figure 5.3. Streamlines of gridded wind data and wind barbs from 54-ft (16.5-m) KSC/CCAS tower 
observations at (a) 1200 UTC 7 June 1996 and (b) 1400 UTC 7 June 1996. Wind speeds are given by 
barbs (open circle = calm, half barb = 5 kt, full barb = 10 kt). The 4-km grid interval used for the 
Barnes objective analysis of zonal (u) and meridional (v) wind components from KSC/CCAS 54-ft 
tower observations is shown in each panel. 

5. 1.1.2 29-km Eta Model Forecast of Sea Breeze Structure 

The 29-km eta model run beginning 0300 UTC 7 June is used to depict the forecast evolution of 
the sea breeze in Figs. 5.5 and 5.6. The left column of Fig. 5.5 (panels a-d) shows 2-m temperature 
gradients and 900-mb vertical velocities while the right column (panels e-h) shows 10-m wind speed 
and direction and divergence of the 10-m wind i(-component. The vertical cross sections of potential 
temperatures, vertical velocities, temperature gradients, and circulation vectors along the lines X-X' 
in Fig. 5.5 are shown in Fig. 5.6. These quantities are displayed at 3-h intervals for the 12-h forecast 
period from 1500 UTC 7 June through 0000 UTC 8 June. 
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The meso-eta model depicts strong differential heating across the land/sea boundaries along the 
east and west coasts of Florida as temperatures are forecast to increase more rapidly over land than 
over water. The evolution of the forecast 2-m temperature gradient is shown by the shading in Fig. 
5.5 (panels a-d) with darker shading indicating stronger temperature gradients. Note that shaded 
temperature gradients are shown in Fig. 5.5 because they provide more remarkable delineation of 
land-sea temperature contrasts than isotherms especially when animating the fields. For the 
purposes of qualitative verification, Table 5.1 lists the observed temperature at Orlando International 
Airport (MCO) and at buoy 41009 located approximately 50 km east of CCAS (see Fig. 5.1 for station 
locations). The forecast temperature gradient along the east coast is most pronounced around 2100 
UTC 7 June (Fig. 5.5c) when the observed temperature difference of 6.6 *C between MCO and 41009 is 
largest (Table 5.1). 



Hour (UTC) 


Figure 5.4. Time series of wind direction from 1100 to 2300 UTC 7 June 1996. The solid line shows 
wind direction at 5-minute intervals averaged over all non-missing points from the gridded analyses 
shown in Fig. 5.3. The dashed line represents the forecast wind direction at XMR from the 29-km eta 
model run beginning at 0300 UTC 7 June 1996. The point forecast data are available at 1-h intervals 
corresponding to the times indicated by the dots along the dashed line. 
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Figure 5.5. Evolution of the forecast sea breeze from the 29-km eta model run beginning 0300 UTC 7 
June 1996. Panels a-d show 2-m temperature gradients (shaded every 1 x 10' 2 K km 1 ) and 900-mb 
vertical velocities (fib s' 1 ) while panels e-h show 10-m wind speed (kt) and direction and divergence of 
the 10-m wind w-component (x 10' 5 s' 1 ). Charts are shown at 3-h intervals from 1500 UTC 7 June 
through 0000 UTC 8 June as labeled in each panel. 10-m wind speed is shaded at 5-kt intervals 
beginning at 5 kt. Solid (dashed) lines in panels a-d indicate upward (downward) motion with an 
isopleth interval of 2 jib s’ 1 . Solid (dashed) lines in panels e-h indicate convergence (divergence) with 
an isopleth interval of 2 x 10' 5 s* 1 . Lines X-X' depict the location of vertical cross sections shown in 
Fig. 5.6. Note that the zero isopleth is omitted on all panels. 
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Figure 5.6. Vertical cross sections of potential temperature (K), vertical velocity (^b s* 1 ), temperature 
gradient (10* 2 K km* 1 ), and circulation along lines X-X' in Fig. 5.5. Potential temperature (K) is shown 
by the thick solid lines while shading shows the horizontal temperature gradient at each level. 
Positive or downward (negative or upward) vertical velocity are indicated by thin solid (dashed) 
lines. Arrows depict circulation in the plane of the cross section. The thick solid lines at the bottom 
of each cross section approximate the location of land along the lines X-X 7 . The isopleth interval is 1 
K for potential temperature and 1 jab s* 1 for vertical velocity. 


Table 5.1. Observed 2-m temperatures (°C) and temperature difference (AT) 
between MCO and buoy 41009 for 7 to 8 June 1996. 


Time (UTC) / 
Date 

Orlando International 
Airport (MCO) 

buoy 41009 

AT (MCO-41009) 

1500 / 7 June 

29.4 

26.7 

2.7 

1800 / 7 June 

32.8 

26.7 

6.1 

2100 / 7 June 

33.3 

26.7 

6.6 

0000 / 8 June 

28.9 

26.7 

2.2 
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In conjunction with the developing temperature gradients shown in Fig. 5.5, the meso-eta 
forecasts a transition in the 10-m wind field that is consistent with the onset of the observed sea 
breeze along Florida's east and west coast. Between 1200 UTC 7 June and 1500 UTC 7 June 1996, the 
model forecast winds along Florida's north-central east coast shift from an offshore direction (not 
shown) to a north-northeasterly onshore direction (Fig. 5.5e). The forecast 10-m winds continue to 
veer from north-northeast at 1500 UTC 7 June to east by 0000 UTC 8 June, especially along the 
northern section of the east coast (Fig. 5.5e-g). A more dramatic shift in direction occurs along the 
west coast where weak east-southeast winds at 1500 UTC 7 June veer by more than 250° and become 
northwesterly by 0000 UTC 8 June as the model forecasts a west coast sea breeze (Figs. 5.5e-h). 

The time series of forecast wind direction at 1-h intervals from XMR (Fig. 5.4) confirms that the 
model veers the wind along the east coast from about 320 to 70° between 1100 and 2300 UTC 7 June. 
The 1-h forecast wind directions are in good agreement with the analyzed, grid-averaged wind 
directions derived from the 5-minute, KSC/CCAS tower observations (Fig. 5.4). From the 5-minute 
temporal resolution of the tower data, it is possible to determine that the onset of the observed east 
coast sea breeze occurs between 1200 and 1400 UTC 7 June as the winds veer from 330 to 60* (Fig. 
5.4). Although the 29-km eta model exhibits a similar but more gradual shift in wind direction 
between 1100 and 1500 UTC 7 June, the forecast winds for this case also veer in agreement with the 
shift of observed winds shown in Fig. 5.4. 

The changes in wind direction associated with the forecast sea breeze are also accompanied by an 
increase in wind speed. The 10-m wind speeds (shading in Figs. 5.5e-h) increase along both coasts 
especially between 1800 and 2100 UTC 7 June. In fact, there is a marked increase in 10-m wind speed 
forecast along the northeast coast as shown by the darker shaded area (speed > 10 kt) moving from 
east of the Florida-Georgia border at 1800 UTC to the northeast coast of Florida by 2100 UTC 7 June 
(Figs. 5.5f, g). Such changes in 29-km eta model 10-m wind speed are useful for identifying the 
occurrence of forecast sea breezes even in cases where the prevailing synoptic flow is already 
onshore. In these cases, there is no distinct shift in wind direction from offshore to onshore flow 
associated with the onset of the sea breeze. However, the winds still accelerate as the pressure 
gradient strengthens in response to stronger heating over land. The forecast evolution of wind speed 
and direction shown in Fig. 5.5 is in general agreement with standard land and buoy surface 
observations (not shown) that confirm the formation of a sea breeze along the east and west coast of 
Florida on 7 June. 

Figure 5.5 shows that the transition to onshore wind flow along both coasts enhances low-level 
convergence and vertical motion, reflecting the development of a thermally-direct circulation over the 
Florida peninsula. These patterns are shown in Fig. 5.5 by the convergence of the n -component of the 
10-m wind (panels e-h) and the 900-mb vertical velocities (panels a-d). Note that the convergence of 
the zonal wind («) rather than total wind is used in order to highlight convergent flows which result 
from changes in wind speed and direction perpendicular to the coastlines. The magnitude of the 10- 
m zonal wind convergence and negative (upward) 900-mb vertical motions increase markedly from 
1500 through 2100 UTC 7 June. The convergence of the 10-m zonal wind exceeds 7 x 10' 5 s' 1 at 2100 
UTC along the northeast coast of Florida (Fig. 5.5g). In addition, the largest maximum upward 
motions on the order of -7 jab s' 1 at 900 mb are forecast over the same area at 2100 UTC 7 June. 

The vertical cross sections along the lines X-X' in Fig. 5.5 (a-d) further illustrate the horizontal 
scale and vertical extent of thermally-direct circulation forecast by the meso-eta model on 7 June. The 
thick solid line at the bottom of each panel in Fig. 5.6 approximates the location of land along the 
lines X-X'. The potential temperatures (K) and shading in Fig. 5.6 show that the 29-km eta model 
forecasts strong temperature gradients below 900 mb along both coasts which is maximized around 
2100 UTC 7 June (Fig. 5.6). The circulation vectors imply low-level convergence especially along the 
west coast as the winds below 900 mb shift from offshore to onshore by 1800 UTC 7 June. The 
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isopleths of forecast vertical motion depict rising motion exceeding -5 pb s' 1 between 975 and 850 mb 
at 0000 UTC 8 June. The core of maximum upward motion occurs over land between 2100 UTC 7 
June and 0000 UTC 8 June with subsidence found offshore along both coasts of Florida (Fig. 5.6). The 
thermally-direct circulation depicted in Fig. 5.6 extends through the lowest 3 km of the model 
atmosphere. The vertical extent of the forecast sea breeze is significantly higher than the 1-2 km 
observed at different mid-latitude and tropical locations around the world (Atkinson 1981, pp. 144- 
145). Local measurements from the KSC Atmospheric Boundary Layer Experiment show the average 
depth of the observed sea breeze at KSC/CCAS to be on the order of 600 to 800 m (Taylor et al. 1990). 

As described in the subjective evaluation protocol (section 3), the KSC/CCAS network of 915- 
MHz wind profilers was to be used for verifying the vertical extent of the sea breeze along the central 
east coast of Florida. However, the 915-MHz profiler data were not available until well after the 
warm season evaluation period ended. As a result, these data can not be used to compare the depth 
of the observed and forecast sea breezes. The evaluation protocol also called for the sea-breeze 
analysis to verify the propagation of the east and west coast sea breeze as inferred from the 
movement of cloud bands shown in Fig. 5.2. This portion of the analysis is not performed primarily 
because the 29-km eta model does not resolve the meso-beta scale structure of the observed sea 
breeze detailed by the cloud patterns and wind fields in Figs. 5.2 and 5.3, respectively. 

5.1.2 Verification of Sea-Breeze Occurrence 

The case example from 7 June 1996 demonstrates that the 29-km eta model can forecast a sea 
breeze that is characterized by a peninsula-scale, thermally-direct circulation. The following analysis 
focuses on determining the skill of the meso-eta model in forecasting the occurrence of an east or 
west coast sea breeze anywhere along the Florida peninsula during the entire warm season (May 
through August 1996). As detailed in section 5.2, the meso-eta model's ability to represent accurately 
the occurrence of convection over Florida is closely related to the forecast sea-breeze circulation 
documented in this section. 

The initial step in the sea-breeze verification is to count the occurrence of forecast and observed 
east and west coast sea breezes. The verification is performed for all days during the warm season 
when (1) both observations and 29-km eta model forecasts are available and (2) either the east or west 
coast of Florida is not affected by tropical waves, tropical cyclones, or fronts. The occurrence of 
observed sea breezes is determined subjectively using the 4-km GOES visible imagery as shown in 
Fig. 5.2. The visible satellite images at 1-h intervals from 1200 through 2300 UTC are animated to 
determine if narrow cloud bands similar to those shown in Fig. 5.2 form along either coast during the 
11-h verification period. As shown in Figs. 5.3 and 5.4, wind analyses from KSC/CCAS tower data 
are also useful to identify the onset of the sea breeze along the central east coast of Florida. However, 
these analyses depict only the meso-beta scale aspects of the observed sea breeze over a small fraction 
of Florida's east coast. Therefore, the 4-km gridded fields of KSC/CCAS tower wind speed and 
direction are animated to confirm the onset of the east coast sea breeze only in cases when the 4-km 
satellite data indicate a sea-breeze event based on cloud patterns. 

Alternatively, it is possible to use Doppler radar data to detect sea breezes and other circulations 
such as horizontal convective rolls as discussed by Weckworth et al (1997). It would be necessary to 
obtain and examine data for the entire warm season from several NEXRAD sites in order to identify 
the occurrence of east and west coast sea breezes along any portion of the Florida peninsula. In terms 
of time and resources, such extensive radar data analysis is beyond the scope of the present study. 
Therefore, verification of the observed sea breezes along both coasts of Florida is based primarily on 
cloud signatures shown by the 4-km visible GOES imagery. 
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There are several limitations with using shallow cloud bands to identify the occurrence of east or 
west coast sea breezes. First, as pointed out earlier in this section, it is likely that sea breeze 
circulations are present prior to the appearance of visible cloud bands. However, the cloud features 
are only used to determine the occurrence rather the actual location and motion of the sea breeze. 
Second, shallow cumulus cloud bands can be obscured by anvil debris or other high-level clouds. On 
some days during the warm season, a wind shift is observed in the KSC/CCAS tower network 
consistent with the onset of a east coast sea breeze but there is insufficient moisture present in the 
atmosphere to form clouds. These few days are excluded from the analysis since the verification 
focuses on the occurrence of east coast sea-breeze events over much broader areas not covered by the 
KSC/ CCAS tower network. 

The occurrence of forecast sea breezes is determined by animating meso-eta gridded fields 
similar to those shown in Figs. 5.5 and 5.6 at 3-h intervals from all available 0300 UTC model runs. 
These fields include 2-m temperature gradients, changes in 10-m wind speed and direction, low-level 
convergence and vertical motion patterns, and thermally direct circulations. For the purpose of 
verification, a forecast east or west coast sea breeze is identified when the sequence of 3-h model 
output shows a shift in wind direction from offshore to onshore flow and/or an increase in wind 
speed along either coast in combination with the other thermodynamic and kinematic features that 
appear in Figs. 5.5 and 5.6. 

The meso-eta model does not have sufficient horizontal resolution to resolve the individual 
circulations associated with the east and west coast sea breezes. Nevertheless, the model is capable of 
forecasting changes in low-level wind speed and direction that are consistent with the onset of the sea 
breeze along either coast as illustrated in Fig. 5.5. In fact, there are a number of warm season cases 
when the model forecasts only an east or west coast sea breeze. Therefore, the analysis considers 
separate verification of east and west coast events. The verification does not include sea-breeze 
forecasts from 1500 UTC model runs because these gridded data were not archived as part of the 
overall 29-km eta model evaluation. 

As shown later in section 5.2, the 29-km eta model often produces low-level convergence and 
thermally direct circulations that are not related to the formation of a sea breeze. Other days are 
characterized by synoptic-scale or mesoscale onshore flow that is not associated with sea-breeze 
circulations and the model forecasts low-level speed convergence as onshore winds decrease due to 
stronger friction over land. For these reasons, forecast sea breezes are more likely to verify correctly 
when the temperature gradients and other characteristic patterns illustrated in Figs. 5.5 and 5.6 are 
present together and oriented nearly parallel to the coast. In conjunction with the gridded model 
fields, the point or station forecasts at XMR and TBW are also used to detect and/or confirm changes 
in wind speed and direction associated with the onset of forecast sea breeze along the east and west 
coast, respectively. 
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The occurrence of forecast and observed east and west coast sea breezes as determined from 
available warm season data are counted and entered in a four-cell contingency table shown in Table 
5.2. The data from the contingency table are then used to compute the bias, false alarm rate (FAR), 
and probability of detection (POD), critical success index (CSI), and Heidke skill score (HSS) for east 
and west coast sea-breeze events. The definitions of the bias, FAR, POD, CSI, and HSS are given in 
Table 5.2 and follow Schaefer (1990) and Doswell et al. (1990). 


Table 5.2. Example of four-cell contingency table used for verification of sea- 
breeze occurrence and definitions of verification scores. 


Forecast Sea Breeze 


Observed Sea Breeze 
Yes No 


Yes 

No 


w 

X 

Y 

z 


N= W+X+Y+Z 
bias = (W + X ) / (W + Y ) 
false alarm rate (FAR) = X / (W + X ) 
probability of detection (POD) = W/ (W + Y) 
critical success index (CSI) = W/ (W + X + Y) 
Heidke skill score (HSS) = [(W + Z)-E]/(N-E) 
E = [(W + Y)(W + X) + (X + Z)(Y + Z)] / N 


The number of forecast and observed east and west coast sea-breeze events and summary 
statistics are presented in Table 5.3. The number of verification days (N=70) is less than the total 
number of days (123) during the warm season evaluation period as a result of missing forecast 
and/or observational data. The bias of 0.63 and POD of 0.56 for east coast events reveals that the 
meso-eta model forecasts the occurrence of the east coast sea breeze just slightly more than 50% of the 
time it is observed. The bias of 0.48 and POD of 0.38 is smaller for west coast than for east coast 
events and indicates that the model underestimates the occurrence of the west coast sea breeze. 
These differences in forecast accuracy between the east and west coast sea-breezes are also reflected 
by the CSI and HSS. For random forecasts the HSS is equal to zero. Therefore, although more 
accurate along the east coast, the model does provide an improvement over random forecasts for the 
occurrence of sea-breezes along both coasts. Finally, the relatively low FAR between 0.10-0.20 
suggest that the model does not often forecast sea breezes which are not observed. 


Table 5.3. East and west coast sea-breeze events and 
summary statistics. 


East Coast Sea Breeze Events | 


Observed 

Bias = 0.63 


Yes 

No 

FAR = 0.11 

Forecast Yes 

24 

3 

POD = 0.56 

No 

19 

24 

CSI = 0.52 




HSS = 0.40 

West Coast Sea Breeze Events | 


Observed 

Bias = 0.48 


Yes 

No 

FAR = 0.20 

Forecast Yes 

16 

4 

POD = 0.38 

No 

26 

24 

CSI = 0.34 




HSS = 0.21 
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There are likely a number of reasons why the 29-km eta model correctly forecasts only slightly 
more than half of observed east and west coast sea breeze events along the Florida peninsula. 
Experience from the AMU warm season forecast exercises suggests that the model typically does not 
forecast the occurrence of observed sea breezes on days characterized by larger-scale forecast errors 
over a significant portion of Florida. For example, the model run beginning at 0300 UTC 18 June 1996 
forecasts Tropical Storm Arthur in the western Atlantic to make landfall north of Daytona Beach, FL. 
As a result, the forecast is dominated by the circulation associated with Arthur rather than the 
characteristic patterns of forecast sea breezes shown in Figs. 5.5 and 5.6. However, Arthur tracked 
more northwesterly and made landfall in South Carolina. While the storm remained to the northeast 
of the peninsula on 18 June, 4-km visible GOES imagery clearly shows the formation of well-defined 
sea breezes along the east and west coasts of Florida (not shown). 

In concluding this section on sea-breeze verification, it is worth noting that 29-km eta model 
forecasts of Florida sea breezes may be of limited utility for the following reasons. 

• The model only resolves the larger-scale aspects of the observed sea-breeze as 
demonstrated by the peninsula-scale, thermally-direct circulation shown in Fig. 

5.6. 

• The model runs beginning at 0300 UTC correctly forecast about 50% of the 
observed east and west coast sea breezes. 

An interesting extension of the sea breeze analysis would be to determine whether model runs 
initialized 12-h later at 1500 UTC provide more accurate forecasts of sea breeze occurrence. However, 
guidance from 1500 UTC model runs is typically available well after the onset of east or west coast 
Florida sea breezes. 

5.2 Thunderstorm Verification 

Warm season thunderstorms in Florida result primarily from interactions between mesoscale 
phenomena. Because larger-scale models commonly used in operations (i.e., NGM, 48-km eta) 
cannot resolve the spatial and temporal details of these phenomena, forecasters must utilize 
observations and persistence to develop accurate short-term (< 6 h) thunderstorm forecasts. Given its 
29-km grid point resolution, the meso-eta model is not expected to resolve features such as individual 
convective cells or thunderstorm outflow boundaries. Although thunderstorms are not explicitly 
forecast by the model, basic diagnostic quantities from its 3-h grid (1-h point) output provide utility 
by allowing users to follow trends and make inferences about the environments which may be 
conducive for thunderstorm development. In order to demonstrate this utility, examples of 
thunderstorm development and verification of warm season precipitation are presented in the 
following sections. 

5.2.1 Case Examples 

The examples presented here are analyses of observed and forecast thunderstorm development 
on 1 to 2 August 1996 (case 1) and 9 to 10 August 1996 (case 2). The first case represents a day when 
the 0300 UTC meso-eta run adequately forecasted several features important for afternoon 
thunderstorm development. These features include surface temperature gradients associated with 
heating in clear versus cloudy areas and low-level wind convergence associated with a developing 
thermal trough. The example from 1 to 2 August presents a best-case scenario since the meso-eta 
model did not forecast the evolution of convection as well for other cases during the warm season. 
Case 2 is chosen to illustrate an alternate scenario for a day when the model produces a much less 
accurate of forecast of convection across Florida. 
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5.2.1.1 Forecast and Observed Convection 


The evolution of forecast and observed convection for cases 1 and 2 are compared using satellite 
imagery and forecast precipitation. Throughout the following discussion, areas of colder cloud tops 
and strong gradients in cloud top temperatures on the 4-km IR satellite images are used as a proxy 
for observed precipitation. The satellite images are chosen at times near the mid-point of the 3-h 
periods over which model precipitation is accumulated. The occurrence of forecast thunderstorms is 
based on total precipitation from 0300 UTC model runs which is accumulated and output on the 40- 
km AWIPS grid every 3 h. The model's total precipitation includes contributions from the convective 
parameterization (Betts and Miller 1986; Janjic 1994) and from the explicit cloud prediction scheme 
(Zhao et al. 1997) that diagnoses rain as part of the cloud microphysics. Similar to sea-breeze 
verification, thunderstorm verification is performed using 0300 UTC rather than 1500 UTC model 
runs. 

A snapshot of the observed and modeled evolution of the events on 1 to 2 August 1996 is 
depicted in Figs. 5.7a-c. The IR satellite imagery at 1915 UTC 1 August (Fig. 5.7a) suggests that 
scattered shower activity is present over northern Florida. At the same time, thunderstorms are 
evident along the east coast sea breeze in southern Florida with fewer clouds over central Florida. 
The forecast precipitation between 1800 and 2100 UTC 1 August appears excessive in northern 
Florida since the 0.25" isopleth covers about one-half of the Florida panhandle (Fig. 5.7a). 
Nevertheless, examination of IR satellite images at 1815 and 2015 UTC 1 August (not shown) reveals 
that at different times within the 3-h forecast period, transient, scattered showers cover the area 
within the 0.10" forecast precipitation isopleth. In this regard, the spatial distribution of observed 
precipitation over northern Florida is forecast reasonably well by the model. In other areas, the 
model adequately forecasts the spatial coverage of the developing thunderstorms over southeastern 
Florida while an absence of forecasted precipitation is noted over east-central Florida (Fig. 5.7a). 

There is one important feature on the 1915 UTC 1 August IR satellite image (Fig. 5.7a) which is 
not forecast by the meso-eta model. The small, bright area to the northwest of Cape Canaveral, FL 
reveals that a thunderstorm is present in the area. Surface weather observations from XMR confirm 
the presence of thunderstorms which produce a 40-kt wind gust at 1819 UTC (Table 5.4). In fact, 
there are a number of other instances throughout the warm season evaluation period when the meso- 
eta model fails to predict the spatial and temporal evolution of individual thunderstorms that 
produce significant weather. 


Table 5.4. Surface weather observations at XMR for 1 August 1996. 


Time 

(UTC) 

Wind Direction/ 
Speed (kt) 

Weather 

Time 

(UTC) 

Wind Direction/ 
Speed (kt) 

Weather 

1655 

210/08 

VCSH 

1855 

90/06 

TSRA 

1755 

260/07 

TS 

1914 

100/02 

TS 

1819 

250/07G40 

TS 

1955 

210/04 

TS 

1833 

120/09 

TSRA 

2039 

150/03 

— 


VC = In the Vicinity; SH = Showers; TS = Thunderstorm; RA = Rain 


At 2215 UTC 1 August, the large area of colder cloud tops in Fig. 5.7b indicates that 
thunderstorms are present across central Florida along a southwest-northeast oriented line. Smaller 
storms continue over the northern and southwestern parts of Florida. Subsequent satellite images 
indicate that convection is still active 3 h later at 0115 UTC (Fig. 5.7c). The meso-eta model 
precipitation shown in Figs. 5.7b, c indicates that the location and spatial extent of the convection 
over central Florida are forecast remarkably well by the model between 1800 UTC 1 August and 0300 
UTC 2 August. As with the previous 3-h period from 1800 to 2100 UTC (Fig. 5.7a), forecast 
precipitation amounts over northern Florida between 0000 and 0300 UTC are still likely high in 
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Figure 5.7. The 4-km GOES infrared imagery for case 1 (1 to 2 August 1996) and case 2 (9 to 10 
August 1996) at times shown in each panel. Meso-eta model forecast precipitation for 3-h periods 
from 1800 to 2100 UTC (panels a, d), 2100 to 0000 UTC (panels b, e), and 0000 to 0300 UTC (panels c, 
f) is shown by the solid lines in each panel. Precipitation amounts are contoured at 0 01 0 10 0 25 
and 0.50". 
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comparison with observed rainfall from inferred from areas of convection depicted on the IR satellite 
image (Fig. 5.7b). 

The sequence of IR satellite images and model precipitation for the second thunderstorm case 
example on 9 to 10 August 1996 is shown in Figs. 5.7d-f. At 1915 UTC 9 August, the primary areas of 
observed convection are found over the Gulf of Mexico, Florida panhandle, south Florida, and over 
the Straits of Florida extending northeast to the northern Bahama islands (Fig. 5.7d). The 3-h 
accumulated model precipitation from 1800 to 2100 UTC shows the 0.01" isopleth covering the areas 
of active convection over south Florida and the Florida panhandle. In addition, heavier amounts 
exceeding 0.10" are forecast in agreement with the broader area of convection oriented southwest- 
northeast in the Straits of Florida (Fig. 5.7d). Except for failing to produce precipitation in association 
with observed convection over the Gulf of Mexico, the model is generating a reasonably accurate 
depiction of the weather at 1915 UTC 9 August. Indeed, the model does not generate precipitation 
over the northern and central parts of the Florida peninsula where skies remain generally cloud free. 

By 2215 UTC 9 August, observed convective activity increases over the Florida panhandle and 
south Florida while it decreases over the Straits of Florida (Fig. 5.7e). The model continues to forecast 
light amounts of precipitation over these same areas as shown by the 0.01" isopleth in Fig. 5.7e. The 
model also develops an extensive area of precipitation along most of Florida's east coast that extends 
into south Georgia with amounts exceeding 0.10" over central east Florida between 2100 UTC 9 and 
0000 UTC 10 August (Fig. 5.7e). This forecast precipitation does not coincide with any observed 
convection at 2215 UTC except the area along the southeast coast to the east of Lake Okeechobee. At 
this point, the model forecast is beginning to diverge from the evolution of events in the real 
atmosphere. 

In the subsequent 3-h period, the IR image for 0115 UTC 10 August (Fig. 5.7f) reveals that the 
cold cloud tops expand dramatically in the Florida panhandle with smaller areas of convection still 
present in south central Florida. The 3-h accumulated precipitation forecast by the model from 0000 
to 0300 UTC 10 August remains light across the Florida panhandle. However, both the spatial 
coverage and amount of precipitation along the east coast increase significantly from the previous 3-h 
period with totals exceeding 0.25" over the eastern half of the peninsula north of Lake Okeechobee 
(Fig. 5.7f). The forecast precipitation from 0000 to 0300 UTC 10 August is erroneous over the central 
and eastern portions of the peninsula since few significant cold cloud tops are evident from the 
satellite imagery in that area at 0115 UTC 10 August (Fig. 5.7f). 

In general, the model's 3-h accumulated forecast precipitation for both cases provides a 
reasonably accurate depiction of the broad-scale areas of observed convection from 1800 to 2100 UTC 
over the domain shown in Fig. 5.7. However, the accuracy of meso-eta precipitation forecasts after 
2100 UTC (1700 EDT) for each case is quite different as discussed in the preceding paragraphs. 
Whereas case 1 depicts a remarkably accurate forecast of developing convection, case 2 represents a 
forecast of excessive precipitation. By exploring differences between these two cases in greater detail, 
lessons may be learned about the meso-eta model's capabilities and limitations in forecasting 
convection. Throughout the AMU's warm season forecasting exercise, it was determined that 
animation of the 3-h gridded model output provides additional value by helping to identify features 
that become important for developing convection. A limited number of model fields from these 
animations are shown in subsequent figures to compare other features of the meso-eta model runs 
that may affect the accuracy of precipitation forecasts for these cases. 
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5 .2.1.2 Forecast 2-m Temperature, 10-m Wind, Vertical Cross Sections 

The 18-h, 21-h, and 24-h forecast 2-m temperatures and 10-m winds at 2100 UTC, 0000 UTC, and 
0300 UTC, respectively along with surface temperature and wind observations are shown for both 
cases in Fig. 5.8. In addition, vertical cross sections of potential temperatures, vertical velocities, 
temperature gradients, and circulation vectors along the lines X-X' and Y-Y' in Fig. 5.8 are depicted in 
Fig. 5.9. These parameters are very similar to those shown for the sea breeze verification as they are 
designed to illustrate the development and vertical extent of thermally-direct circulations produced 
by the model on 1 to 2 August and 9 to 10 August 1996. ' 

Forecast 2-m temperatures (’C) at 2100 UTC 1 August indicate that central Florida is warmer than 
surrounding land areas to the north and south (see shading in Fig. 5.8a). The forecast temperature 
gradient between warmer and cooler areas likely resulted from differential heating between the 
relatively cloud-free area over central Florida and the cloudy regions in the northern and southern 
sections of the peninsula. Corresponding surface temperature observations (’C) are plotted as small 
numbers in Fig. 5.8a and corroborate the forecast temperature distribution. As forecast surface 
temperatures exceed 31 C, a thermal trough (not shown) develops in the surface pressure field in an 
orientation perpendicular to line X-X'. Streamlines of the forecast 10-m winds suggest a southwest- 
northeast line of low-level convergence associated with the developing thermal trough (Fig. 5.8a). 
The forecast streamlines in Fig. 5.8a clearly illustrate the onshore flow and convergence associated 
with the thermal trough although they do not exactly match the observed wind directions shown by 
the wind barbs plotted in Fig. 5.8a. 

By 0000 UTC 2 August, the forecast 2-m temperature gradient weakens as cooling begins to occur 
over land areas near sunset. Surface temperature observations plotted in Fig. 5.8b confirm this 
forecast trend, particularly over central Florida where decreases in temperatures are greatest. 
However, the low-level convergence zone oriented southwest-northeast across the peninsula is still 
well-defined by the forecast 10-m streamlines at this time. Here again the low-level flow pattern is in 
general agreement with the observed winds although discrepancies exist especially at coastal stations 
in south Florida (Fig. 5.8b). The 2-m temperature gradient continues to weaken across Florida at 0300 
UTC 2 August and is primarily found along the coastline as the land areas cool faster than the 
adjacent water bodies. The observed surface temperatures at this time range from 24 to 25 *C in 
central Florida and also indicate a very weak temperature gradient from north to south across the 
region (Fig. 5.8c). The convergence in forecast 10-m winds at 0000 UTC 2 August across central 
Florida is replaced by a general southeast flow at 0300 UTC 2 August that becomes more southerly to 
southwesterly over northern sections of the state. The model forecasts the larger-scale aspects of the 
10-m wind field quite well at 0300 UTC 2 August although there are still some substantial local 
differences compared with the observed wind directions shown in Fig. 5.8c. 

The sequence of forecast and observed 2-m temperatures and 10-m streamlines for 2100 UTC 9 
August to 0300 UTC 10 August (case 2) is shown in Figs. 5.8d-f. As with case 1, the strongest thermal 
contrast between land and the surrounding waters occurs at 2100 UTC and decreases with time as 
daylight hours end. As the temperature contrast intensifies, the model forecasts the development of a 
thermal trough (not shown) and associated area of convergence that is indicated by the 10-m 
streamlines in Figs. 5.8d-f. This low-level convergence zone slowly propagates westward from the 
east coast of Florida at 2100 UTC 9 August to the center part of the peninsula by 0300 UTC 10 August. 
For the most part, the forecast patterns of low-level temperature and wind patterns from 2100 UTC 9 
to 0300 UTC 10 August are supported by the surface temperature and wind observations plotted in 
Figs. 5.8d-f. The forecast temperatures and streamlines shown in Fig. 5.8 reveal that both cases 
develop low-level convergence zones which are supported by observations. It is useful to examine 
vertical cross sections in order to compare the strength of circulations that form in response to the 
low-level convergence. 
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Figure 5 8. Meso-eta model forecasts of 2-m temperature (shading in "C) and 10-m wind streamlines 
for case 1 (1 to 2 August 1996) and case 2 (9 to 10 August 1996) at times shown in each panel. 
Corresponding surface temperature C’C) and wind (kt) observations are plotted using standard 
convention. Lines X-X' and Y-Y' depict the location of vertical cross sections shown in Fig. 5.9. 
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Vertical cross sections for case 1 along the lines X-X' in Figs. 5.8a-c are shown in Figs. 5.9a-c. The 
forecast potential temperature (K) and shading of the temperature gradient in the model highlight the 
fact that the strongest thermal contrast occurs at 2100 UTC 1 August over central Florida in the lowest 
model layers below 900 mb (Fig. 5.9a). In addition, the shading around 700 to 600 mb in Fig. 5.9a 
su gg ests the presence of a lower tropospheric temperature inversion. The circulation vectors in Fig. 
5.9a imply low-level convergence. The isopleths of vertical motion show rising motion over the 
center of the surface convergence zone and area of maximum heating while subsidence occurs on 
either side. The maximum upward motion on the order of -4 pb s' 1 at 2100 UTC 1 August is centered 
around 850 mb (Fig. 5.9a). 

By 0000 UTC 1 August, the low-level temperature gradients are more diffuse (Fig. 5.9b) which is 
consistent with the decrease in 2-m temperatures shown in Fig. 5.8b. However, the vertical velocities 
increase from -4 pb s' 1 at 2100 UTC 1 August to -6 pb s' 1 . As upward motion extends above 500 mb 
the weak temperature inversion evident at 2100 UTC between 600 and 700 mb (shading in Fig. 5.9a) is 
not present across the entire cross section at 0000 UTC 1 August (Fig. 5.9b). At 0300 UTC 2 August, 
the low-level thermal gradients continue to weaken but the vertical velocities increase to more them -7 
pb s' 1 between 750 to 650 mb (Fig. 5.9c). The patterns of horizontal and vertical motion for case 1 
(Figs. 5.9a-c) depict a thermally-direct circulation which is likely enhanced in response to developing 
forecast precipitation across central Florida after 2100 UTC 1 August (Figs. 5.7b, c). 

The vertical cross sections for case 2 also show the development of a thermally-direct circulation 
(Figs. 5.9d-f). As with case 1, the most-pronounced low-level temperature gradients occur at 2100 
UTC. In fact, the thermal gradients in the lower troposphere and those associated with the 
temperature inversion around 650 mb are stronger for case 2 than for case 1. A comparison of 
vertical velocities in Figs. 5.9b and 5.9e clearly shows that upward motions at 0000 UTC are also 
stronger for case 2 than for case 1. However, this trend does not continue at 0300 UTC as the 
maximum rising motion for both cases is on the order of -7 pb s' 1 around 700 mb. 

5.2.1.3 Forecast and Observed Sounding Parameters 

Examination of the precipitable water (PWAT) and convective available potential energy (CAPE) 
from forecast and observed soundings at XMR and TBW are shown to illustrate differences between 
the moisture and stability for cases 1 and 2 that may indirectly contribute to the aforementioned 
strength of the vertical circulations. Although forecast soundings at XMR and TBW are available 
every hour, stability parameters are only compared for those times listed in Table 5.5 when 
corresponding observed soundings are available. Note that the 2200 UTC soundings correspond to 
the time when forecast precipitation develops near XMR in both cases as shown in Figs. 5.7b, e. For 
case 1, the model underestimates the CAPE at all times for both stations which is consistent with the 
negative bias in CAPE identified during the entire warm season (Table 4.1; section 4.3). At TBW, the 
forecasts of PWAT agree very well with observed values. However, forecasts soundings at XMR are 
too dry as indicated by the smaller PWAT compared with the observed values in Table 5.5. 
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Figure 5.9. Vertical cross sections of potential temperature (K), vertical velocity, temperature 
gradient, and circulation along lines X-X' and Y-Y' in Fig. 5.8. Potential temperature (K) is shown by 
the thick solid lines while shading emphasizes the horizontal temperature gradient at each level (x 
10' 2 K km’ 1 ). Positive or downward (negative or upward) vertical velocity (pb s' 1 ) are indicated by 
thin solid (dashed) lines. Arrows depict circulation in the plane of the cross section. The isopleth 
interval is 1 K for potential temperature and 1 pb s' 1 for vertical velocity. The temperature gradients 
are shaded every 1 x 10 2 K km 1 . 
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Table 5.5. Precipitable water (PWAT; mm) and convective available potential energy 
(CAPE; J kg 1 ) derived from forecast (FCST) and observed (OBS) soundings at XMR and TBW 
for 1 to 2 August 1996 and 9 to 10 August 1996. Dashes denote missing data. 


Date 

Time 

(UTC) 

XMR (Case 1) 

Date 

Time 

(UTC) 

XMR (Case 2) 1 

PWAT 

FCST OBS 

CAPE 

FCST OBS 

PWAT 

FCST OBS 

CAPE 

FCST OBS 

1 Aug 

1000 

41 

47 

13 

2070 

9 Aug 

1000 

40 

40 

255 

1353 

1 Aug 

2200 

50 

54 

1446 

1493 

9 Aug 

2200 

50 

44 

1916 

1232 

2Au s 

1000 

49 

— 

1 

— 

10 Aug 

1000 

41 

38 

2 

2025 



TBW (Case 1) 



TBW (Case 2) 


Time 

PWAT 

CAPE 


Time 

PWAT 

CAPE 

Date 

(UTC) 

FCST 

OBS 

FCST 

OBS 

Date 

(UTC) 

FCST 

OBS 

FCST 

OBS 

1 Aug 

1200 

47 

47 

158 

1007 

9 Aug 

1200 

46 

52 

54 

1042 

2 Aug 

0000 

48 

48 

736 

2134 

10 Aug 

0000 

45 

50 

1660 

1754 

2Au s 

1200 

49 

— 

85 

— 

10 Aug 

1200 

49 

47 

21 

1897 


The examination of forecast and observed CAPE and PWAT for case 2 indicates similar 
differences with one very important exception. At XMR, the forecast sounding at 2200 UTC is 
actually too moist with a PWAT of 50 mm compared with the observed value of 44 mm. In addition, 
the forecast CAPE of 1916 J kg* 1 is more than 50% larger than the observed value of 1232 J kg' 1 (shaded 
row in Table 5.5). Since available upper air observations over the peninsula are sparse, it is difficult to 
determine if similar errors in PWAT and CAPE are present over broader regions along the northeast 
coast of Florida where forecast precipitation for case 2 appears excessive after 2100 UTC 9 August 
(section 5. 2. 1.1). Nevertheless, excess moisture and instability present in the forecast sounding at 
XMR is probably sufficient to trigger the convective parameterization and eventually produce 
precipitation. 


5.2.1.4 Case Summary 

In summary, there are similarities and differences between the model forecasts for cases 1 and 2. 
First, the 2-m temperature gradient and 10-m convergence zone for case 2 are oriented parallel to the 
coast and result from differential low-level heating across the land/sea interface. In fact, the 
thermally-direct circulation identified on 9 to 10 August is nearly identical to that shown for the case 
example of the forecast sea breeze in section 5.1. This result is not surprising since the overall warm 
season sea-breeze verification indicates that the model correctly forecasts the occurrence of an east 
coast sea breeze on 9 August 1996. In contrast, the low-level thermal and convergence patterns in 
case 1 likely form in response to differential heating between clear versus cloudy regions (Fig. 5.7a). 
Although the model correctly forecasts the occurrence of an east coast sea breeze for this case as well, 
the thermally-direct circulation for 1 to 2 August is not directly associated with the formation of a sea 
breeze. 

A thermally-direct circulation is forecast by the model for both cases. In fact, the low-level 
temperature and convergence patterns identified in association with these circulations are found for 
numerous warm season cases. On some days such as 1 to 2 August 1996 (case 1), forecast 
precipitation coincides reasonably well with observed patterns of convection. On other days, the 
model overestimates the coverage and amount of convective precipitation as shown for 9 to 10 
August 1996 (case 2). During the warm season, the PWAT and CAPE biases (Table 4.1) suggest that, 
on average, forecast soundings are too dry and too stable. In contrast, the model soundings for case 2 
at XMR are too moist and unstable just prior to the onset of forecast precipitation over the northeast 
portion of the Florida peninsula. Both surface observations at XMR (not shown) and IR satellite data 
indicate that this forecast precipitation is erroneous. 
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It is possible that inaccurate precipitation forecasts for case 2 result from excessive moisture and 
instability documented in the forecast sounding at XMR. It would be useful to determine why the 
model overestimates the moisture and instability for case 2 when similar errors are not apparent for 
case 1. One plausible explanation is that too much Atlantic moisture is transported into the region by 
the low-level convergent flow associated with the thermally-direct circulation. Furthermore, errors in 
moisture flux could be related to inaccurate forecasts of wind speed and direction and moisture. It is 
apparent that more detailed diagnostics of model output and additional observations are required to 
explain differences in convection forecasts between cases such as those described in this section. A 
logical extension of the thunderstorm verification would be to examine more warm season cases to 
determine if errors similar to those discussed for case 2 affect the model’s convective forecasts. 
Although such an analysis is beyond the scope of the current evaluation, it would likely help model 
users to understand more thoroughly the capabilities and limitations of warm season, meso-eta 
model precipitation forecasts over Florida. 

5.2.2 Verification of Precipitation Occurrence 

The case example from 1 to 2 August 1996 demonstrates that the meso-eta model occasionally has 
some utility in forecasting thunderstorms over Florida. However, there are a number of forecasts 
during the warm season that were much less accurate as shown for the case from 9 to 10 August 1996. 
The following analysis focuses on the skill of the model in forecasting thunderstorms over regions of 
the Florida peninsula during the entire warm season (May to September 1996). 

5.2.2.1 Methodology 

There are several steps involved in the verification of precipitation occurrence. First, the state is 
divided into six verification zones shown in Fig. 5.10. The motivation for bisecting the state from 
north to south is to determine if the meso-eta model could forecast distinct areas of convection 
associated with the east and/or west coast Florida sea-breeze circulations. The remaining divisions 
along the 27 and 29' latitude lines are subjective and result in six zones of roughly equal area (not 
counting the western most area of the panhandle). It is interesting to note that the width of each zone 
over land is approximately 120 km. Since the 29-km eta model can only resolve features with 
wavelengths on the order 4Ax or 116 km, the zone width corresponds well with the model's smallest 
resolvable wavelength. 



Figure 5.10. Map of Florida showing definition of precipitation verification zones. See text for details. 


62 




The next step in the precipitation verification is to count the occurrence of forecast and observed 
precipitation over land in each zone during 3-h periods from 1500 to 1800 UTC, 1800 to 2100 UTC, 
and 2100 to 0000 UTC. These time periods are chosen to verify the forecast occurrence of 
thunderstorms during the 9 h from 1100 to 2000 EDT (1500 to 0000 UTC) when convection is most 
often observed in Florida during the warm season. The occurrence of forecast thunderstorms is 
determined using meso-eta gridded fields of 3-h accumulated total precipitation and is based on total 
precipitation values exceeding 0.01" (0.254 mm) anywhere in a zone during the 3-h period. 
Experience from the warm season forecast exercises suggests that more than 95% of the total 
precipitation over Florida is generated by the convective parameterization. The verification of 
forecast precipitation occurrence is determined using only gridded data from the 0300 UTC 
initialization of the model since the gridded data from 1500 UTC initialization were not archived as 
part of the overall meso-eta model evaluation. 

The occurrence of observed thunderstorms is determined subjectively from all available 4-km 
visible (VIS) and IR GOES satellite data and from surface weather reports of rain and thunderstorms. 
The VIS and IR satellite images at every hour from 1500 to 0000 UTC are animated to locate distinct 
anvils in the VIS data and strong gradients of cold cloud top temperatures in the IR data. These 
features are used as a proxy for the occurrence of precipitation in each zone during the 3-h periods. It 
is likely that anvil debris, cirrus clouds, or the lack of well-defined anvils or cold cloud tops affect the 
accuracy of this satellite-based, subjective technique in delineating areas of actual precipitation. 
Therefore, routine surface airway observations of precipitation (i.e. rain or thunderstorms) are 
included to account for such deficiencies. It is important to point out that surface weather 
observations alone probably underestimate the occurrence of precipitation given the nonuniform 
distribution and relative coarse spacing of the stations. In cases where either satellite data or surface 
observations are missing for the entire 3-h period, the remaining available data are used to identify 
observed thunderstorms. When both satellite data and surface observations are available, observed 
thunderstorms are identified in the 3-h window if either data type indicates their presence based on 
the criteria discussed above. 

The occurrences of all forecast and observed thunderstorms as determined from available warm 
season data in the six zones over each 3-h period are counted and entered in four-cell contingency 
tables similar to those shown in Table 5.2. In cases where forecast or observed precipitation is located 
on zone boundaries or across adjacent zones, these events count as "yes" occurrences for each zone 
containing the specified area of precipitation. The data from these contingency tables are then used 
to compute the bias, FAR, and POD for each zone and time period. The definitions of the bias, FAR, 
and POD are given in Table 5.2. 

The methodology used in this study for the verification of precipitation occurrence differs from 
traditional methods. The traditional methods used to verify precipitation involve a point-by-point 
comparison of forecast and observed amounts at selected thresholds (e.g. 0.01", 0.25", etc.) over the 
model grid (e.g. Olson et al. 1995). The statistics such as bias, FAR, POD, and threat score or critical 
success index derived from such analyses do not usually account for spatial errors in forecasting 
precipitation. On the other hand, these methods often verify forecast and observed precipitation over 
periods of 12 to 24 h which allows for significant errors in forecasting the temporal evolution of 
precipitation. 

The technique used is this study focuses on the occurrence of precipitation (> 0.01") anywhere 
within a region on the order of 100 km x 200 km. As illustrated in section 5.2.1. 1, the 29-km eta model 
does not resolve small-scale thunderstorms and associated precipitation. However, experience from 
the warm season forecast exercises suggests that broad areas of model-generated precipitation could 
be subjectively correlated with precipitation that was observed over much smaller sections of the 
same area. By performing a zone assessment of forecast and observed precipitation occurrence, a far 
less stringent test of model capabilities is applied than traditional methods which measure exact 
spatial correlation between forecast and observed precipitation. However, precipitation forecasts in 
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the current evaluation are verified over 3-h periods to highlight temporal errors and determine 
whether the model can forecast the observed diurnal cycle in warm season precipitation over Florida. 
In this regard, the evaluation strategy is more stringent than that used in traditional methods 
especially when precipitation is verified over much longer time periods (e.g. 24 h). 

5.2.2.2 Results 

Summary statistics for each of the zones are presented in Table 5.6. When all available data are 
pooled together the sample size of valid forecast/observed data ranges from 73 to 78 depending on 
zone number and 3-h verification period. Since the sample size is relatively small, it is difficult to 
determine if subtle differences between the scores in each zone are statistically significant. The 
following discussion focuses on scores from zone 5 that covers east central Florida and includes XMR 
(Fig. 5.10). 

Within the first 3-h period from 1500 to 1800 UTC, a bias of 1.76 indicates that forecast 
precipitation occurs 76% more often than actually observed. In later periods, the bias improves to 
1.00 between 1800 to 2100 UTC before increasing slightly to 1.21 between 2100 to 0000 UTC. As the 
bias scores improve, the FAR scores correspondingly decrease. Between 1500 to 1800 UTC, the FAR 
in zone 5 is 0.52, indicating that the occurrence of precipitation is incorrectly forecast on 52% of valid 
days. In later periods, the FAR drops to a more respectable 0.30 (1800 to 2100 UTC) and 0.27 (2100 to 
0000 UTC). While the bias and FAR show improvement with time, the POD fluctuates without any 
clear trend. Since values of POD in zone 5 are > 70%, the model tends to accurately forecast the 
occurrence of most observed rain events. However, these values must be viewed in context with 
values of bias and FAR; a high POD is only effective when the corresponding FAR is low and the bias 
near unity. Since the contingency table includes observed precipitation regardless of whether it can 
be resolved by the 29-km eta model, the POD for all zones and time periods may improve (i.e. 
approach unity) by verifying forecast and observed precipitation only at scales which are resolved by 
the model. 


Table 5.6. Summary statistics for the verification of precipitation occurrence within each 
of six zones shown in Fig. 5.10. See Table 5.2 for definitions of bias. False Alarm Rate (FAR), 
Probability of Detection (POD), and Heidke Skill Score (HSS). 


Zone 1 

15-18 UTC 



Zone 4 

15-18 UTC 

18-21 UTC 

21-00 UTC 

Bias 

2.44 


0.98 

Bias 

1.57 

0.91 

1,28 

FAR 

0.62 


0.23 

FAR 

0.52 

0.31 

0.37 

POD 

0.94 

0.73 

0.75 

POD 

0.76 

0.63 

0.81 

HSS 

0.52 

0.48 

0.66 

HSS 

0.48 

0.53 

0.64 

Zone 2 

15-18 UTC 

18-21 UTC 

21-00 UTC 

Zone 5 

15-18 UTC 

18-21 UTC 

21-00 UTC 

Bias 

2.05 

1.05 

1.31 

Bias 

1.76 

1.00 


FAR 

0.60 

0.24 

0.32 

FAR 

0.52 

0.30 


POD 

0.81 

0.79 

0.89 

POD 

0.84 

0.70 

0.89 

HSS 

0.60 

0.61 

0.78 

HSS 

0.61 

0.58 

0.73 

Zone 3 

15-18 UTC 

18-21 UTC 

21-00 UTC 

Zone 6 

15-18 UTC 

18-21 UTC 

21-00 UTC 

Bias 

1.74 


1.04 

Bias 

1.69 

0.86 

1.00 

FAR 

0.53 

0.19 

0.21 

FAR 

0.53 

0.18 

0.27 

POD 

0.81 

0.71 

0.82 

POD 

0.79 

0.71 

0.73 

HSS 

0.65 

0.64 

0.78 

HSS 

0.68 

0.66 

0.72 


Examination of the contingency tables used for the analysis (not shown) reveals that precipitation 
was observed (forecast) in zone 5 on 32% (57%), 51% (51%), and 58% (70%) of valid days during the 
periods 1500 to 1800 UTC, 1800 to 2100 UTC, and 2100 to 0000 UTC, respectively. A comparison of 
these percentages indicates that the bias approaches unity during the last two times periods due to 
increases in the frequency of observed precipitation. In general, the 0300 UTC initialization of the 
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meso-eta model forecasts precipitation too frequently during the first 3-h period from 1500 to 1800 
UTC (1100 to 1400 EDT). However, within the later afternoon and early evening periods from 1800 to 
2100 UTC and 2100 to 0000 UTC, the model shows more utility in delineating whether precipitation is 
likely to be observed in a specific zone. These results are supported by the HSS, which indicates that 
the model provides the greatest improvement over random forecasts (i.e., HSS = 0) in the later time 
period. While this discussion only focuses on scores for zone 5, Table 5.6 shows similar results for the 
other zones. 

For the period from 2100 to 0000 UTC, the bias (FAR) of 1.28 (0.37) and 1.22 (0.27) in zones 4 and 
5, respectively indicates that the model overestimates the occurrence of precipitation along the central 
and northeast coast of Florida during the entire warm season. These results are consistent with the 
example shown in case 2 where model precipitation is forecast from 2100 UTC 9 to 0000 UTC 10 
August in zones 4 and 5 but is not supported by surface and IR satellite data. The analysis of 
precipitation occurrence for the entire warm season is designed to quantify added value to the user 
by determining how often the meso-eta model produces precipitation forecasts such as those shown 
for case 1 versus case 2 in section 5.2.1. Such verification is useful since conclusions drawn about 
model capabilities in forecasting warm season precipitation over Florida are limited by examining 
only two cases which essentially represent the best and worst of meso-eta model capabilities. 
Moreover, the verification provides a summary of the times and areas over which the meso-eta model 
is most capable of generating useful convection forecasts given the wide variety of different scenarios 
encountered during the warm season. 

5.3 Tropical Wave Verification 

The capability of the meso-eta model to forecast the development and movement of warm season 
tropical disturbances was documented as part of the daily forecast exercises. For the purposes of this 
evaluation, tropical disturbances (or tropical waves; TW) include both easterly waves (Riehl 1954, 
Carlson and Lee 1978) and tropical upper tropospheric troughs (TUTT; Sadler 1975, Whitfield and 
Lyons 1992) that typically move from east to west and are easily identified by the motion of an 
organized and distinct area of enhanced convection. Although Atlantic tropical storm and hurricane 
activity was above-average during the summer of 1996, there are few TW which entered the subset 
model domain (e.g. the area shown in Fig. 5.1). In fact, during the warm season forecast exercises 
only seven potential TW cases were documented which were not tropical storms or hurricanes. 

A post-seasonal examination of forecast graphics and satellite imagery reveals that only one of 
these cases persisted within the subset model domain for longer than 2 subsequent forecast periods. 
Given only one case example, no conclusions should be drawn regarding how accurately the 29-km 
eta model forecasts the development and movement of TW. However, experience from the warm 
season forecast exercises suggests that persistence of wave features such as a cyclonic turning of the 
lower tropospheric winds or weak sea-level pressure troughs between subsequent model runs for 2 
or 3 days is important for classifying potential TW. In particular, the meso-eta model often generates 
small-scale waves which propagate around the southwestern edge of the large-scale sub-tropical 
anticyclone. These features have typical wavelengths on the order of 100's of km while the horizontal 
scale of TW in the lower troposphere is on the order of 1000 km. Although these small-scale waves 
sometimes help focus developing convection across Florida, they should not be interpreted as TW 
due to their small size and lack of persistence between subsequent model runs. 

5.4 Cold Front Verification 

Fronts are usually identified as a line of confluent winds that precede narrow zones of tight 
gradients in temperature and/or dew point temperature (Wallace and Hobbs 1977). Cold fronts in 
particular represent the leading edge of a transition zone to a relatively colder, drier air mass. An 
extensive review of theoretical and observational studies on fronts is provided by Bluestein (1986), 
Keyser (1986), and Carlson (1991). While mesoscale sea-breeze fronts are common during the warm 
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season in Florida, synoptic-scale cold fronts are frequent during the cool season. These synoptic- 
scale, cool season cold frontal passages through central Florida are commonly associated with locally 
adverse weather which may include extensive precipitation, low visibility, strong horizontal and 
vertical wind shear, and cold temperatures. Moreover, the environment associated with synoptic- 
scale cold fronts contains instabilities that are conducive for the intensification of mesoscale weather 
phenomena (Keyser 1986). Given these possibilities, meso-eta forecasts of synoptic-scale cold frontal 
passages through east-central Florida (hereafter referred to as fronts, or cold fronts) are compared 
with observations for the cool season period October 1996 through January 1997. A single case 
example is presented which considers some aspects of the meso-eta model's capabilities and 
limitations in depicting small-scale details of cold frontal passages. Finally, a summary is provided 
which compares the observed and forecast timing of frontal passages at XMR throughout the cool 
season evaluation period. 

5.4.1 Case Example 

The example presented here is an analysis of a forecast and observed cold frontal passage 
through central Florida on 8 to 9 November 1996. The meso-eta forecast for this case is from the 0300 
UTC cycle on 8 November 1996 with frontal passage at XMR occurring later that day near 2200 UTC. 
Animation of diagnostic quantities for the meso-eta forecast grids such as vorticity, vertical velocity, 
convergence, moisture and temperature advection, convective parameters, etc., provide useful ways 
to depict the forecast evolution of the atmosphere. Unfortunately, the sparsity of observational data 
limits verification of the forecasts, particularly over offshore areas to the west and east of the Florida 
peninsula. Use of the KSC/CCAS mesonet data for verification of this case is not appropriate 
because the width of the transition zone associated with the forecast synoptic-scale cold front is larger 
than the area covered by the mesonet. For these reasons only general, subjective comparisons 
between the forecast and observations are made in terms of cloud cover, precipitation, low-level 
winds and dew point temperatures, and upper air and surface data at XMR. 

5.4.1.1 Clouds and Precipitation 

On 8 November 1996, a surface cold front associated with a developing cyclone was oriented 
generally in a northeast to southwest direction and extended through the Carolinas, Georgia, and 
northern Florida. As the front propagated through central Florida, forecast total cloud fraction and 
3-h total precipitation amounts are compared with 4-km IR satellite imagery and observed 
precipitation composites (Fig. 5.11). Model forecast cloud fraction depicts the percentage of cloudy 
area in a grid box and is estimated using relative humidity within all model layers (Zhao et al. 1997). 
Hourly precipitation composites are derived by NCEP using a combination of available Office of 
Hydrology rain gauge observations and NEXRAD-derived precipitation estimates (Baldwin and 
Mitchell 1996). The precipitation amounts shown in Fig. 5.11 are derived by summing the hourly 
NCEP composite amounts over the 3-h forecast period. Note that observed precipitation amounts 
shown in Fig. 5. Ilf represent only a 2-h accumulation because the 2200 UTC precipitation estimate is 
not available. 

At 1800 UTC 8 November, the model depicts a northeast to southwest oriented band of clouds 
approximately 500 km wide moving across the Florida peninsula (Fig. 5.11a). Cloud fractions of 
100% are forecast to exist in a narrow band from around Jacksonville (JAX) to Tampa Bay (TBW), FL 
and southwest into the Gulf of Mexico. In conjunction with this narrow band of 100% cloud cover, 
3-h forecast precipitation amounts between 1500 and 1800 UTC exceed 0.75" from near JAX to TBW. 
In comparison, 4-km IR imagery (Fig. 5.11b) reveals that observed clouds cover approximately the 
same area as depicted by the forecast cloud cover. In particular, scattered cloud cover is observed 
over south Florida and over most areas of the Atlantic where forecast cloud fractions are generally 
less than 70%. Observed clouds appear more dense in the satellite imagery along the same line where 
the model predicts 100% cloud fractions. Observed precipitation is heaviest along a line that is 
slightly northwest of the axis of maximum precipitation forecast by the model at 1800 UTC 8 August 
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Figure 5.11. Forecast total cloud fraction and 3-h precipitation in panels a-c with corresponding 4-km 
GOES IR imagery and estimates of observed 3-h precipitation in panels d-f. Charts are shown at 3-h 
intervals from 1800 UTC 8 November through 0000 UTC 9 November as labeled in each panel. 
Forecast total cloud fraction is shaded from 0% (darkest) to 100% (lightest) at intervals of 10%. The 3- 
h forecast and observed precipitation is accumulated from 1500 to 1800 UTC, 1800 to 2100 UTC, and 
2100 to 0000 UTC with isopleth intervals at 0.01, 0.10 (dashed), 0.25, 0.50 and 0.75". In panel (f), 
observed precipitation is accumulated for 2 h due to missing data at 2200 UTC. Station identifiers for 
Jacksonville, FL (JAX) and Tampa Bay, FL (TBW) are shown in panel (f). 
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(compare Figs. 5.11a, b). Locally intense 3-h observed accumulations exceed 1.0" off the west coast of 
Florida. 

At 2100 UTC, the area of forecast and observed clouds and precipitation propagates to the 
southeast by about 100 km (Figs. 5.11c, d). The heaviest forecast 3-h precipitation amounts across 
central Florida are again in excess of 0.75". To the northeast of TBW, observed 3-h accumulations 
(Fig. 5. lid) are locally in excess of 0.25". Again, the area covered by precipitation is forecast 
remarkably well but the model cannot predict the small-scale details. Model cloud fraction forecasts 
continue to be in agreement with the satellite imagery at 2100 UTC. Cloud fractions of 100% remain 
collocated with the area where brighter cloud tops are observed across central Florida. 

The area of clouds and precipitation associated with this cold front continued to move to the 
southeast through 0000 UTC 9 November (Figs. 5. lie, f). However, the previously banded structure 
in the precipitation field becomes more intricate as it breaks apart into smaller sections. While 
observed 3-h rainfall amounts between 1800 and 2100 UTC locally exceed 0.25" across central Florida, 
2-h accumulations in the same area reach just 0.10" between 2100 and 0000 UTC. Meanwhile, a new 
band of heavy precipitation formed offshore over the Atlantic with 2-h rainfall amounts in excess of 
1.0" (Fig. 5. Ilf). As is consistent with all previous case examples, the meso-eta model is not capable 
of resolving the small-scale details of these changes in the cold frontal rainband structure. However, 
the model does predict a general decrease in precipitation amounts across central Florida with a 
corresponding split in the primary cold frontal rainband (Fig. 5.1 le). 

Examination of hourly point data for this case indicates that 0.36" of rainfall was forecast by the 
model at XMR, mostly between 2000 and 0000 UTC. Surface observations at XMR reveal that a light 
rain shower at 2104 UTC produced merely 0.06" of rain. Although the forecast amount of rainfall is 
excessive at XMR, the forecast grids shown in Fig. 5.1 le indicate that amounts could have been much 
greater if the model did not accurately forecast the decrease in intensity and split in the rain band. 
The model forecast is able to predict the larger-scale area covered by cloud and precipitation for this 
case with remarkable accuracy. 

5.4.1.2 Winds and Dew Point Temperatures 

While the animation of cloud cover and precipitation forecasts are useful for identifying the 
extent and motion of weather associated with cold frontal passages, surface data are required to track 
the position of the cold front. The parameters used to identify the position of the frontal zone near 
the surface usually include winds, pressure (heights), moisture, or temperature. For this particular 
case example, winds and dew point temperatures provide the strongest representation of frontal 
position. In Fig. 5.12, forecast 1000-mb dew point temperatures and 10-m wind streamlines are 
plotted along with available surface observations at 2100 UTC 8 November 1996. 

For this example, a line of sharply confluent winds appears in the forecast streamlines to the east 
of the Florida peninsula (Fig. 5.12). This line precedes the leading edge of rapidly decreasing forecast 
dew point temperatures across central Florida. A heavy line on the figure corresponds to the location 
of the cold front inferred by forecast wind direction and dew point temperatures. Observations of 
wind and dew point temperature plotted on the figure agree reasonably well with the corresponding 
forecast parameters. In particular, wind directions generally match those indicated by the forecast 
streamlines although discrepancies exist especially along Florida's southeastern coast. Observed dew 
point temperatures are within about ±2 °C of forecast values and clearly indicate a sharp decrease to 
the northwest across central Florida. Based on observed dew point temperatures and wind 
directions, the position of the leading edge of the observed frontal zone also appears to lie close to the 
forecast frontal position (heavy line in Fig. 5.12). 


68 



Examination of wind directions and dew point temperatures indicate that the forecast and 
observed frontal zones appear to be nearly collocated at 2100 UTC 8 November 1996. However, 
closer examination of Fig. 5.12 reveals that there are differences in the horizontal scale of the forecast 
and observed frontal zones. In particular, the forecast wind direction shift appears to be more 
gradual than observed. Forecast winds shift from southwest just offshore Cape Canaveral to 
northwest along the west coast of Florida. A similar wind shift is observed in the observations over 
the much shorter distance between Titusville (TIX) and Orlando International Airport (MCO) in east 
central Florida. Although the model appears to forecast the location of the frontal zone quite 
accurately across central Florida, the 29-km grid point resolution is not adequate to capture the 
sharpness of the observed cold front. 



Figure 5.12. Forecast 10-m wind streamlines and 1000-mb dew point temperatures (°C) with available 
observations of surface wind (kt) and dew point temperatures (°C) at 2100 UTC 8 November 1996. 
Forecast dew point temperatures are shown in gray shading every 2 *C with darker shading 
representing drier air as indicated on the temperature scale. Long (half) barbs represent 10 (5) kt 
wind observations with numerals shown for dew point temperature observations. Station identifiers 
are given for MCO and TIX. 
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5.4.13 Point Forecasts at XMR 


It is possible to identify the time when the leading edge of the frontal zone passes through XMR 
using hourly surface data. Before using meteograms of hourly surface data to identify frontal 
passages at a point, it is necessary to note the existence of the approaching front in the horizontal 
distribution of forecast and/or observed data. When considered alone, time series of surface 
parameters sometimes exhibit shifts which could be misinterpreted as frontal passages. For the 8 to 9 
November 1996 case, a front has already been identified using precipitation, cloud fraction, and 
surface wind and dew point temperature {Figs. 5.11, 5.13). The figures and corresponding discussion 
demonstrate that the forecast and observed cold fronts stretch across central Florida around 2100 
UTC and are clearly identifiable by the dew point temperature gradient and wind shift. 

Forecast and observed meteograms of temperature, dew point temperature, mean sea-level 
pressure (MSLP), wind speed, and wind direction at XMR are shown in Fig. 5.13 for the 33-h period 
from 0300 UTC 8 November through 1200 UTC 9 November 1996. At 2200 UTC, observed dew point 
temperatures begin to decrease rapidly at the rate of about 5 *C hr -1 (Fig. 5.13b). Forecast dew point 
temperatures also begin decreasing at that time, though at a slower rate relative to observations. A 
sudden shift in observed wind direction (Fig. 5.13e) is also observed at 2200 UTC. The corresponding 
forecast wind shift is more gradual, yet clearly evident. The meteograms of dew point temperature 
and wind direction (Figs. 5.13b, e) indicate that the leading edge of the forecast and observed frontal 
zone passes through XMR around 2200 UTC 9 November 1996. This is not surprising given the 
forecast and observed position of the front previously noted in Fig. 5.12 using wind shifts and dew 
point temperature gradients. 

Other parameters displayed in the meteograms (Fig. 5.13) do not show as clear a depiction of 
when the front arrives at XMR. For example, forecast and observed temperatures begin to decrease 
prior to the frontal passage simply in response to the normal diurnal cooling near sunset. Wind 
speeds in this example show a steady increase with time, but there is no indication of a sudden jump 
in wind speed which sometimes accompanies a cold frontal passage. The time of frontal passage at 
XMR may not be inferred directly from MSLP but increasing values at 2200 UTC lend support to the 
fact that pressures are rising as a ridge builds in behind the advancing cold front. 

It is interesting to note that forecast changes in meteogram parameters occur more slowly than 
observed. For example, at 2200 UTC, the observed winds at XMR shift from about 210 to 300* in one 
hour. The same shift in forecast wind direction occurs gradually over four hours from about 2000 
UTC 8 to 0000 UTC 9 November. As discussed in section 5.4.1.2, similar changes in wind direction 
also occur over a shorter horizontal distance in the surface observations than in the forecast 
streamlines. Therefore, hourly meteogram data also indicate that the meso-eta model does not 
resolve the sharp gradients of temperature, moisture, wind, etc. at the scales which are characteristic 
of observed frontal zones. 

The passage of the cold front through XMR is also apparent in forecast and observed profiles of 
upper air temperature, dew point temperature, and winds. The forecast and observed soundings at 
XMR 10-h prior, 2-h after, and 14-h after frontal passage are depicted in Figs. 5.14a, b, and c, 
respectively. The soundings reveal that the model generally is able to handle the increase in moisture 
as clouds and precipitation enter central Florida and the subsequent drying that occurs after frontal 
passage. Moreover, the depth of the post-frontal intrusion of cold air is evident by the low-level 
temperature inversion and wind shifts that develop in the lower 200 mb of both forecast and 
observed soundings (Fig. 5.14c). 
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Figure 5.13. Forecast and observed meteograms at XMR of 2-m temperature (panel a) , 2-m dew point 
temperature (panel b), mean sea-level pressure (MSLP; panel c), 10-m wind speed (panel d) and 10-m 
wind direction (panel e). The meteograms cover the 33-h period from 0300 UTC 8 November through 
1200 UTC 9 November 1996. Solid lines depict forecast values from the 0300 UTC meso-eta model 
run, dotted lines depict observed values, and the vertical dashed lines indicate the apparent time of 
frontal passage through XMR at 2200 UTC 8 November. The horizontal dashed line in panel e is 
added for emphasis. 
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Figure 5.14. Skew-t plots of forecast and observed sounding data at XMR valid (a) 1200 UTC 8 
November 1996, (b) 0000 UTC 9 November, and (c) 1200 UTC 9 November. All forecasts are taken 
from the model run initialized 0300 UTC 8 November 1996. Heavy solid (dashed) lines are forecast 
(observed) temperature and dew point temperatures fC). Forecast (observed) wind barbs (kt) are 
shown in bold (normal) to the right of each skew-t plot. 

Many of the differences between the forecast and observed variables shown in Figs. 5.13 and 5.14 
are consistent with the cool season objective verification of those parameters. For example, dew point 
temperatures and wind speeds following cold frontal passage are greater than observed (Fig. 5.13b, 
d). A similar positive bias appears in the statistics shown in Figs. 4.2b and 4.4b. The forecast 
temperature inversion that develops near 700 mb after frontal passage (Fig. 5.14c) is at a higher level 
than observed. This result is consistent with the cool season biases shown in Fig. 4.15a which depict a 
sharp cool bias at around the same level. In spite of these few notable difficulties, the meso-eta 
forecasts of most parameters for this case of frontal passage are reasonably accurate. Again, this 
result is consistent with the fact that biases for most parameters discussed in section 4 are relatively 
small. 
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5.4.2 Cold Frontal Timing at XMR 


The case example discussed in section 5.4.1 demonstrates some of the capabilities and limitations 
of the meso-eta model in forecasting cold frontal passages through central Florida. However, not all 
forecasts of frontal passages are quite as accurate. Many cases are much more complicated and 
involve phenomena such as pre-frontal rain bands, developing cyclogenesis, and stalled fronts. In an 
attempt to quantify the overall accuracy of cold front forecasts in central Florida, the timing of cold 
frontal passages through XMR is documented throughout the cool season period. 

In order to perform a verification of frontal passages at XMR, all available 0300 UTC forecast 
grids are examined for the presence of fronts. As discussed earlier, it is difficult to identify frontal 
passages using meteograms alone without prior knowledge that a front exists in the domain. 
Therefore, 0300 UTC meso-eta forecasts of MSLP, 10-m winds, 850-mb vertical velocity, 2-m dew 
point temperature, and 1000-mb frontogenesis are all used to help identify the movement of cold 
fronts across central Florida. Frontogenesis is easily derived from forecast winds and potential 
temperature using GEMPAK and is used to quantify the rate of increase of the magnitude of the 
temperature gradient with time (Carlson 1991, p. 351). Once the frontal passage events are identified, 
forecast and observed meteograms are studied over the corresponding periods to determine the time 
of frontal passage to the nearest hour. 

On each of the days in which a frontal passage is evident in the forecast grids, hourly point 
forecast and observed surface data at XMR are plotted as meteograms similar to those shown in Fig. 
5.13. Although all five variables are considered simultaneously, shifts in wind direction to the west 
or northwest and rapid decreases in dew point temperatures are primarily used to identify the frontal 
passage at XMR. Shifts in temperatures, MSLP, and wind speed are occasionally useful to help 
support the identification of frontal passage based on dew point temperature and wind. Results of 
forecast and observed frontal passages through XMR are summarized in Table 5.7. 

A total of ten observed fronts pass through XMR during the cool season period on days for which 
both gridded and point forecast data and observed surface data at XMR are available. Several other 
cold fronts entered northern Florida, however, they are not counted in this summary because 
observations indicate that the fronts never actually reach XMR. Of the cases considered in Table 5.7, 
more than half of the forecast cold fronts passed through XMR within one hour of the observed 
frontal passage. For the 0900 UTC 19 December event, the cold front is not evident at the end of the 
33-h forecast cycle beginning 0300 UTC 18 December, as indicated by the word "None" in Table 5.7. 
Overall, timing accuracy does not appear to depend on forecast duration. Model runs initiated at 
0300 UTC are slightly more accurate in forecasting cold frontal passage at XMR than those initiated at 
1500 UTC. However, this result may be due to the small number of cases sampled during this cool 
season period. 

The timing and motion of cold fronts through XMR has been documented for the cool season 
evaluation period. In addition, a limited case study has been performed for one front which passed 
through XMR at 2200 UTC 8 November 1996 in both the forecast and observations. Cool season 
results reveal that on many days the model is capable of forecasting frontal passages at XMR to the 
nearest hour. The case example demonstrates that many smaller details of the frontal passage are 
simulated reasonably well, including the area covered by clouds and precipitation, and the location 
of the frontal zone in surface parameters. Animation of meso-eta model output may provide users 
with substantial added value in terms of forecasting the timing, movement, and intensity of weather 
moving through central Florida in association with cold frontal passages. However, it is important to 
note that the 29-km resolution of the eta model is not sufficient to resolve the sharpness of observed 
gradients in temperature, wind, and moisture associated with cold frontal zones. 
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Table 5.7. Time of forecast and observed frontal passages (FROPA) through XMR for 
the 1996-1997 cool season evaluation period. Forecast duration indicates the length of the 
forecast in hours when the forecast front passes through XMR. Timing errors are calculated 
in hours as observed passage minus forecast passage. In this manner, negative (positive) 
timing errors indicate that forecast fronts arrived at XMR too late (early). For some days, 
frontal passages are listed multiple times because the same front is apparent in subsequent 
model runs. 


Observed 
FROPA at XMR 

0300 UTC Forecast Cycle 

1500 UTC Forecast Cyc 

ie 

FROPA at XMR 

Error 

(hours) 

FROPA at XMR 

Error 

(hours) 

Date 

Time 

(UTC) 

Date 

Time 

(UTC) 

Duration 

(hours) 

Date 

Time 

(UTC) 

Duration 

(hours) 

19 Oct 

0500 

19 Oct 

0900 

30 

-4 

Not available 





19 Oct 

0600 

3 

-1 





3 Nov 

0200 

2 Nov 

2300 

20 

+3 

3 Nov 

0300 

12 

-1 

8 Nov 

2200 

8 Nov 

2200 

19 

0 

8 Nov 

2200 

31 

0 







8 Nov 

2300 

8 

-1 

22 Nov 

1200 

22 Nov 

1300 

10 

-1 

22 Nov 

1400 

24 

-2 

26 Nov 

1700 

26 Nov 

1700 

14 

0 

26 Nov 

1900 

4 

-2 

2 Dec 

0900 

2 Dec 

0900 

30 

0 

2 Dec 

1200 

21 

-3 



2 Dec 

0900 

6 

0 





8 Dec 

1000 

8 Dec 

0900 

6 

1 

8 Dec 

1200 

21 

-2 

19 Dec 

0900 

None 

30 


19 Dec 

1300 

22 

-4 



19 Dec 

1000 

7 

-1 





9 Jan 

2300 

10 Jan 

0200 

23 

-3 

10 Jan 

0400 

13 

-5 

16 Jan 

1700 

16 Jan 

1900 

16 

-2 

16 Jan 

1800 

27 

-1 







16 Jan 

2300 

8 

-6 


5.5 Verification of Warm and Cool Season Winds 

The original evaluation protocol called for the subjective verification of warm season steady-state 
winds exceeding 18 kt and selected warm or cool season case studies of mesoscale wind features 
identified from time-height sections of 50-MHz and/or 915-MHz profiler data. During the 1996 
warm season, the standard, hourly airways observations at XMR indicate winds > 18 kt on six days 
primarily in response to nearby thunderstorms. The 0300 UTC meso-eta model runs archived during 
the warm season did not forecast winds > 18 kt at XMR. Furthermore, the point forecasts from the 
1500 UTC model runs indicate winds > 18 kt on only two occasions, neither of which correlate with 
observed convective events. In general, the meso-eta model forecasted wind > 18 kt during either 
season only in the presence of strong pressure gradients associated with tropical and/or extratropical 
cyclones, fronts, etc. For this reason, no additional verification is performed to assess the utility of the 
meso-eta model in forecasting convective winds > 18 kt during the warm season. 

The evaluation protocol specifies that hourly time-height wind forecasts at XMR were to be 
verified using 50-MHz and 915-MHz profiler data and wind observations from the KSC/CCAS tower 
network. However, the 915-MHz profiler data were not routinely archived until after the end of cool 
season evaluation period. Although 50-MHz profiler data were archived, no analysis of wind events 
is possible below the lowest gate of the profiler at roughly 2 km. On the other hand, the KSC/CCAS 
towers sample only the lowest levels of the atmospheric boundary layer near the surface. Finally, 
model users indicated during the pre-final report discussions that results obtained from one or two 
cases would be too limited unless they could be generalized for the entire season as done for the sea- 
breeze and thunderstorm verification. Based on these problems with data and potential limitations 
with the analyses, case studies of mesoscale wind features were not performed. 
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5.6 Verification of Cool Season Cloud Forecasts 

The selection of cool season cloud cases was based on more challenging forecasts such as the 
development of stratocumulus clouds that occurs frequently in the cool season following the passage 
of a cold front. The evaluation was designed to determine if the 29-km eta model provides any added 
value for such forecasts verified using surface and launch reconnaissance aircraft observations and 
XMR rawinsonde relative humidity profiles. As with the wind analysis, there was concern expressed 
during the pre-final report discussions that limited verification of explicit cloud water and/or relative 
humidity forecasts from one or two cases is not sufficient to reveal overall model capabilities in 
forecasting clouds. Since a more detailed and complete verification of meso-eta model cloud 
forecasts over the warm and/or cool season is beyond the scope of the current evaluation, no cases 
studies of cloud forecasts were performed. 
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6.0 Summary and Lessons Learned 


This section summarizes results from the objective verification of meso-eta model point forecasts 
and the subjective verification of sea breezes, thunderstorms, and cold fronts. The section concludes 
with a general summary and lessons learned from the overall meso-eta model evaluation. 

6.1 Summary of Objective Verification 

The objective verification of the eta model focused on the overall accuracy of wind, temperature, 
and moisture forecasts at XMR, TBW, and EDW for the warm and cool seasons. The statistical 
measures used to quantify model forecast errors are the bias, RMS error, standard deviation, and 
consistency. Using these statistics, point forecasts from the 0300 UTC and 1500 UTC meso-eta model 
cycles are verified against standard surface and rawinsonde observations. Convective parameters 
and 850 to 500-mb layer-averaged wind and relative humidity are derived from the forecast 
soundings and verified against corresponding values from observed soundings. As specified in the 
evaluation protocol, results were stratified by the 950- to 600-mb layer-averaged wind direction. 
However, examination of results for every parameter indicates that error characteristics are 
qualitatively similar under both westerly and easterly flow regimes. For this reason, all available 
data are combined and final results are not stratified by wind regime. 

6.1,1 Surface Parameters 

Overall, results of the surface parameter verification reveal that forecast errors are on average 
reasonably small. However, there are a few identifiable biases which include overestimation of dew 
point temperature and wind speed during the cool season at XMR and diurnal changes in warm 
season temperature, wind speed, and MSLP at EDW. 

The only benchmark specified in the evaluation protocol is a comparison of 10-m winds with 1 to 
6-h persistence. Results of this benchmark reveal that 1- to 3-h persistence forecasts of wind speed 
and direction usually have smaller RMS errors than the corresponding meso-eta model forecasts. 
However, the model forecasts of these variables are occasionally more accurate than 6-h persistence. 

In general, RMS error trends for most parameters reveal that the model exhibits minimal error 
growth throughout the 33-h forecast period. The error variance often comprises a large portion of the 
total error. This suggests that errors in surface parameters are commonly due to more random, non- 
systematic variations in the forecasts and/or observations. Consistency results indicate that 
subsequent model runs tend to agree more closely with one another than with observations. Since 
the magnitude of errors depends on parameter, location, and season, it is difficult to specify whether 
the model is generally more accurate in forecasting surface parameters during the warm or cool 
season. 


6.1.2 Upper Air Parameters 

Examination of results for all upper air parameters reveals that errors are qualitatively similar for 
both the 0300 and 1500 UTC forecast cycles. Therefore, sounding data from both model cycles are 
combined while performing the verification of upper air parameters. As with surface variables, 
forecast errors for upper air parameters are on average reasonably small. RMS error trends among 
each of the three available verification periods indicate that the model exhibits minimal error growth. 
Consistency results again suggest that subsequent model runs tend to agree more closely with one 
another than with observations. 
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For many parameters, a large portion of error standard deviations may be explained, in part, by 
rawinsonde measurement uncertainty. As with surface forecasts, error variances provide large 
contributions to the total error. There are a few identifiable biases which include difficulties in 
resolving tropopause heights and cool season lower tropospheric temperature inversion heights at 
XMR and TBW. In addition, warm (cool) season forecast errors at XMR and TBW indicate a cool, dry 
(warm , moist) bias which suggests that forecast soundings are on average more (less) stable than 
observed. 


6.1.3 Convective Indices and 850 to 500-mb Layer-Averages 

During the warm season, negative biases in precipitable water (PWAT) and convective available 
potential energy (CAPE) and positive biases in the lifted index (LIFT) indicate that forecast soundings 
are typically drier and more stable than observed. These errors are consistent with lower 
tropospheric warm season biases in mixing ratio and temperature at XMR which indicate that 
forecasts tend to be thermodynamically more stable than observed. During the cool season, positive 
biases in CAPE and the K index (KINX) and negative biases in LIFT indicate that cool season 
forecasts are more unstable than observed. These results are also consistent with the low-level warm 
bias in temperature and moist bias in mixing ratio found during the cool season at XMR. In general, 
the results of both convective parameter and 850 to 500-mb layer-average verifications are consistent 
with characteristic biases identified in the upper air statistics. 

Errors in forecast convective parameters on any given day may actually be large enough to 
provide misleading information regarding the likelihood for thunderstorm development. In 
particular, errors in any given convective index result from errors in moisture, temperature or wind 
variables which are used to compute the index and may be cumulative (in the vertical), especially for 
integrated quantities such as CAPE. Therefore, it is important to understand the accuracy and 
relationships of all variables which are used to derive each index. 

6.2 Summary of Subjective Verification 

The subjective verification of sea breezes, thunderstorms, and cold fronts is designed to assess the 
added value of the meso-eta model in forecasting selected aspects of these phenomena. The 
evaluation strategy consists of limited cases and seasonal verification. Seasonal verification 
quantifies the added value and is important because conclusions drawn about model limitations and 
capabilities in forecasting the aforementioned phenomena are limited by examining only a few cases. 

6.2.1 Sea Breezes 

The analysis of the forecast and observed sea-breeze development from 7 June 1996 demonstrates 
that the meso-eta model forecasts a sea breeze that is characterized by a peninsula-scale thermally 
direct circulation. The thermally direct circulation is driven by differential heating across the 
land/ sea boundaries along the Florida peninsula. The features associated with the forecast sea 
breeze identified from 3-h model output include a thermal trough in sea-level pressure over the 
peninsula, shift in 10-m level wind direction from offshore to onshore flow, and low-level 
convergence and vertical motion patterns oriented parallel to the coastlines. Based on the results 
from this single case, the meso-eta model appears to depict an evolution of the sea breeze that is 
dynamically realistic although at a larger scale than observed. It is important to note that the 29-km 
horizontal resolution of the model is not sufficient to resolve the individual circulations associated 
with east or west coast sea breezes. Instead, the model generates a single low-level convergence zone 
and an associated circulation that generally lies parallel to the coastlines. 
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The sea-breeze verification is also designed to determine how reliably the meso-eta model 
forecasts the occurrence of east or west coast sea breezes anywhere along the Florida peninsula 
during the entire warm season. This portion of the analysis is important because previous studies 
have shown that the timing and location of convection over Florida is modulated by interactions 
between the sea-breeze circulations and synoptic-scale flow. The results indicate that the 0300 UTC 
model runs correctly forecast the occurrence of sea breezes about 50% of the time they are observed 
during the warm season. The utility of sea-breeze forecasts may be limited for this reason and the 
fact that the meso-eta model resolves only the larger-scale aspects of the observed sea breeze. In 
some cases, the failure of the model to forecast the occurrence of the sea breeze is likely due to larger- 
scale forecast errors over a significant portion of Florida. 

6.2,2 Thunderstorms 

Two cases are presented of forecast and observed thunderstorm development on 1 to 2 August 
1996 (case 1) and 9 to 10 August 1996 (case 2). The first example shows a best-case scenario when the 
meso-eta model depicts a remarkably accurate forecast of developing convection in the late-aftemoon 
and early evening across the Florida peninsula. The second case illustrates an alternative scenario 
when the model forecasts excessive precipitation along much of Florida's east coast during the same 
time period. The results from these and other warm season cases (not shown) suggest that the model 
generates broader areas of organized convection which, on occasion, are remarkably accurate. In 
contrast, there are a number of instances during the warm season when the model did not forecast 
areas of organized convection or the evolution of individual thunderstorms that produced significant 
weather. This result is expected because the 29-km horizontal resolution of the meso-eta model is too 
coarse to resolve convection at those scales. 

Since a wide variety of scenarios for the development of forecast and observed convection were 
observed during warm season forecast exercises, a verification of precipitation occurrence is required 
in order to quantify the utility of the model in forecasting warm season convection. Given the 
limitations in resolving small-scale convection, traditional precipitation verification using point-to- 
point comparison at selected thresholds was not done. Instead, verification of precipitation over 3-h 
periods in zones on the order of 100 km x 200 km is performed for all available warm season days. 
The technique does not specifically address precipitation verification within 25 miles of XMR as 
specified in the original evaluation protocol. However, it does provide a means to quantify the 
accuracy of the model in forecasting the occurrence of larger areas of organized convection without 
requiring that the model produce the correct amount of precipitation at exactly the location where it 
is observed. 

The bias in all zones over Florida from 1500 to 1800 UTC ranges from 1.69 to 2.44 indicating that 
the meso-eta model forecasts precipitation to occur more often than observed. During later time 
periods from 1800 to 2100 UTC and 2100 to 0000 UTC, the bias, POD, and FAR in all zones indicate 
that the meso-eta model shows more utility than in the earlier period (1500 to 1800 UTC) in 
successfully delineating whether precipitation will occur in a specific zone. The statistical scores such 
as bias for zone 5 improve with time in part due to an increase in frequency of observed precipitation 
within that zone. Since observed precipitation is counted in the contingency tables regardless of 
scale, the statistics may improve further by excluding observed precipitation events such as isolated 
thunderstorms which can not be resolved by the model. 

The verification of warm season precipitation occurrence indicates that the model generates 
excessive precipitation from 1500 to 1800 UTC despite the fact that biases in convective parameters 
such as CAPE and LIFT show that 1000 UTC forecast soundings are on average typically drier and 
more stable than observed. At least 95% of the warm season forecast precipitation over Florida is 
produced by the convective scheme in the meso-eta model. It is possible that forecast soundings 
destabilize too rapidly after 1000 UTC thereby triggering the convective parameterization which 
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eventually produces precipitation more often than it is actually observed. For example, the erroneous 
precipitation forecast in case 2 may be related to excessive moisture convergence and instability as 
documented by a comparison of forecast and observed soundings at XMR along the east coast. 
However, more comprehensive analyses of model output and additional observations are required to 
diagnose the reasons for both accurate and inaccurate forecasts of convective precipitation 
throughout the warm season. 

6.2.3 Cold Fronts 

The analysis of a cold front which passed through central Florida on 8 to 9 November 1996 
demonstrates some of the meso-eta model's capabilities and limitations in depicting the timing and 
motion of cold frontal passages. The case reveals that although the model is not able to predict the 
small-scale details, it is capable of forecasting the larger-scale areas covered by clouds and 
precipitation with remarkable accuracy. Moreover, the location of the leading edge of the frontal 
zone near the surface is in agreement with observations of winds and dew point temperatures. 
Although the model appears to forecast the location of the frontal zone quite accurately across central 
Florida, the 29-km grid point resolution is not adequate to capture the sharpness of the observed cold 
front. 

Hourly plots of 2-m dew point temperature and 10-m wind direction for this case reveal that the 
leading edge of both forecast and observed frontal zones pass through XMR within the same hour. It 
is interesting to note that shifts in forecasts of surface parameters are more gradual than 
corresponding shifts in observed data as the front passes XMR. This result supports the idea that the 
meso-eta model does not resolve the sharp gradients of surface temperature, moisture, wind, etc. at 
the scales which are characteristic of observed frontal zones. Forecasts of most parameters for this 
case of frontal passage are generally quite accurate, a result which is consistent with the relatively 
small biases noted earlier for the entire cool season. In fact, a verification of cold front timing at XMR 
over the entire cool season reveals that the model is accurate to within the nearest hour for a majority 
of documented cold frontal events. 

6.3 Overall Evaluation Summary and Lessons Learned 

The evaluation described in this report is designed to assess the utility of the meso-eta model for 
local weather forecasting in support of 45WS, SMG, and NWS MLB operational requirements. In 
conclusion, the following points summarize overall results from the AMU's year-long evaluation. 

• In general, objective verification results reveal that meso-eta model point 
forecasts at XMR, TBW, and EDW are reasonably unbiased. This result suggests 
that the model has few substantial systematic errors and on average, can be used 
reliably. However, there are some exceptions identified in this evaluation as 
indicated by the following list of model biases. 

2-m dew point temperatures and 10-m wind speeds are typically 
overestimated at XMR during the cool season. 

Diurnal changes exist in the average forecast errors for 2-m temperature, 10- 
m wind speed and MSLP at EDW. 

At XMR and TBW, warm (cool) season forecast soundings are typically drier 
and more stable (unstable) than observed. 

The height of the lower tropospheric inversion at XMR and TBW is 
misrepresented during the cool season. 
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Tropopause heights are misrepresented by the model at all three stations. 

• Objective verification results also indicate minimal error growth with time based 
on RMS errors. This means that, on average, forecast accuracy does not vary 
substantially throughout the 33-h forecast period. 

• The error variance for many variables comprises a large portion of the total RMS 
error. In these instances, total model error over the course of an entire season is 
dominated by the day-to-day variability in forecasts and/or observations. 

Subjective verification of sea breezes, thunderstorms, and cold fronts is very important to 
quantify added value of the model forecasts for these specific phenomena which can not be readily 
inferred from statistics over many cases (i.e. from objective verification). Moreover, subjective and 
objective verification are complimentary and results from each component of the evaluation are 
generally consistent. Some results from the subjective evaluation which can be important for 
operational forecast concerns include the following. 

• The forecast sea breeze is characterized by a peninsula-scale, thermally direct 
circulation that forms in response to differential heating across the land/sea 
boundaries along the Florida peninsula. The occurrence of sea breezes are 
correctly forecast about 50% of the time they are observed during the warm 
season. The 29-km grid point resolution of the model is not sufficient to resolve 
the individual circulations associated with the observed west and/or east coast 
sea breezes. 

• Two case studies demonstrate different situations where the model generates 
both accurate and inaccurate forecasts of larger scale, organized convection. 
However, the model's 29-km grid point resolution is not sufficient to accurately 
forecast the development of isolated thunderstorms. 

• Cold frontal passages through XMR are often forecast to within the nearest hour 
of observed passages. The spatial and temporal evolution of weather associated 
with frontal passages is also depicted well by animation of the 3-h forecast 
gridded products. However, the ability of the model to represent small scale 
details such as the width of the surface frontal zone is limited by the model's 29- 
km resolution. 

• Results from the objective verification do not indicate whether the model is more 
accurate overall during either the warm or cool season. However, results from 
the subjective verification suggest that the model forecasts over central Florida 
may be more useful during the cool season. This statement is based on the fact 
that the meso-eta model resolution is not yet sufficient to resolve the small-scale 
details of sea and river/lake breeze circulations, thunderstorm outflow 
boundaries, and other phenomena which play a dominant role in determining 
the short-term evolution of weather over east central Florida during the warm 
season. 

• Objective verification results also demonstrate that forecasts of selected 
parameters are reliable over the course of an entire season. On the other hand, 
results from the subjective verification demonstrate that model forecasts of 
developing weather events such as thunderstorms, sea breezes, cold fronts, etc. 
are not always as accurate as implied by the seasonal error statistics. 
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The AMU's daily real-time warm season forecast exercise proved to be a valuable component of 
the overall subjective verification because it revealed how operational forecasters could use the 0300 
UTC cycle of the meso-eta model for local forecasting. Lessons learned from these daily weather 
discussions are as follows. 

• Animation of 3-h model output with color enhancements and overlay of multiple 
fields (winds, temperature, etc.) is useful to identify features and trends that 
could become important for developing weather (as illustrated with sea breeze, 
thunderstorm, and cold front case examples). 

• Availability of digital gridded model output at 3-h intervals is important because 
it gives users the flexibility to select variables, cross sections, overlay options, 
contour intervals etc. and it provides the temporal resolution needed to track 
specific aspects of forecast weather events such as the timing of cold frontal 
passage, onset of sea breeze, etc. 

• The model often generates small scale vortices that are difficult to confirm with 
observations. Many times, these vortices are not realistic and lead to fluctuations 
in point forecast variables such as wind direction/speed and convective 
parameters (as demonstrated in the large error standard deviations of these 
variables). 

• In order to exploit the four-dimensional capability of the meso-eta and other 
models in forecasting possible realizations of the atmosphere, sufficient 
communication bandwidth and computer processing power are necessary to 
retrieve, process, and examine output data. This requirement will become more 
important in the future as NCEP increases the number of meso-eta model runs 
per day, the model resolution, and potentially the frequency of model output. 



7.0 Future AMU Efforts on 29-km Eta Model Evaluation 


The evaluation methodology used for this study is comprised of both objective and subjective 
components stratified over two 4-month evaluation periods. While this strategy enables an 
assessment of both forecast accuracy and utility for model users, it also imposes some limitations on 
the completeness of the verification. In particular, limited samples sizes make it difficult to perform 
various event- and regime-based stratifications and to apply statistical significance tests to the data. 
Moreover, results could be affected by interannual variability which can not be determined from a 
single one-year analysis. 

Subjective verifications in this evaluation are limited to simple case studies in order to examine a 
greater number of phenomenological features which impact 45WS, SMG, and NWS MLB operations. 
The case studies highlight some of the capabilities and limitations of the meso-eta model but are not 
designed to fully explore the cause(s) of model errors. Seasonal evaluations of sea breezes, 
thunderstorms, and cold front timing are designed to quantify how consistently the model provides 
utility in forecasts of these phenomena. However, more detailed examination of model capabilities 
and limitations is beyond the scope of the present evaluation given available AMU resources. 

As part of ongoing efforts to improve the accuracy and utility of forecast products, NCEP is 
continuing to update the configuration, initialization, and physical parameterizations of the meso-eta 
model. These updates could modify some of the seasonal error characteristics identified by the 
AMU's objective verification of surface and upper air forecasts at XMR, TBW, and EDW. In order to 
increase the sample size and track possible changes in model accuracy, the objective component of 
the meso-eta evaluation is being extended to include a second warm and cool season period from 
May through August 1997 and October 1997 through January 1998, respectively. An extension of the 
subjective component of the evaluation would also be useful and has been discussed as part of the 
AMU's annual tasking process. However, it was determined that such an evaluation can not be 
continued effectively within the scope of the AMU's available resources. 

As part of the ongoing meso-eta model evaluation, comparisons will be made for the objective 
verification results between the two warm season and the two cool season evaluation periods at 
XMR, TBW, and EDW. A comparison between results from the 1996 and 1997 seasons will highlight 
any changes in the error characteristics at these stations which may occur in response to updates in 
the meso-eta model configuration. This analysis is also useful for model users since the 1997/1998 
results will be more representative of the meso-eta model's current capabilities. 
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Appendix 

Objective Verification Results Listed as a Function of Height 

As specified in the original evaluation protocol and described in section 3.2, objective verification 
results are listed as a function of geopotential height. During the pre-final report discussions, it was 
stated that wind variables are of primary interest for applications to shuttle flight rules and should be 
listed in tabular format for direct reference. Plots of wind speed and direction errors as a function of 
height (not shown) exhibit exactly the same characteristics as those shown in Figs. 4.18 through 4.21. 
Therefore, the discussions in sections 4.2.4 and 4.2.5 are applicable to the wind speed and direction 
errors listed below. Although results for u and v wind components are not discussed in previous 
sections they are generally consistent with results for wind speed. As discussed in section 4, 
substantial differences do not appear to exist between the 0300 and 1500 UTC forecast cycles. 
Moreover, the statistics do not reveal any obvious differences when the results are stratified by the 
950- to 600-mb layer-averaged wind direction. For these reasons, all available data from each season 
and verification period (see section 3.2 and Fig. 3.1) are included in the statistical calculations below. 
Results from TBW are not shown but are in fact very similar to those shown below for XMR. 
Standard deviations are not shown because they can be calculated from the bias and RMS errors 
using Eq. 4 and because they were not requested in the original evaluation protocol. Sample sizes 
collected at XMR and EDW are listed in Table A.9 as a function of height for each season. At EDW, 
results below 3000 ft are not available since the lowest level of data extracted from model point 
forecasts at that location is at an elevation of 3238 ft. 
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Table A.l. Warm season biases for wind variables at XMR as a function of geopotential height (kft). 
See Fig. 3.1 for definitions of verification periods 1, 2, and 3. 


Verf. Period 
Height (kft): 

U-wind (m s' 1 ) 
12 3 

V-wind (m s' 1 ) 
12 3 

Wind Speed (m s' 1 ) 
12 3 

Wind Direction (°) 
12 3 

i 

-0.1 

0.1 

0.0 

-0.5 

-0.6 

0.4 

-0.1 

-0.2 

0.2 

-11 

-8 

-3 

2 

-0.8 

-0.2 

-0.2 

0.3 

-0.5 

1.1 

0.2 

0.2 

0.7 

-8 

-3 

-5 

3 

-1.4 

-0.5 

-0.8 

0.2 

0.1 

1.1 

0.6 

0.2 

0.8 

-2 

1 

-5 

5 

-2.0 

-0.6 

-0.9 

0.6 

0.4 

1.2 

0.5 

0.0 

0.2 

-1 

-12 

-13 

10 

-0.8 

0.0 

0.2 

-0.3 

-0.4 

-0.6 

0.2 

0.0 

0.4 

-1 

-7 

-1 

12 

-0.7 

-0.1 

0.2 

-0.4 

-0.4 

-0.4 

-0.1 

-0.1 

0.0 

2 

-2 

-4 

20 

-1.3 

-0.5 

-0.6 

-0.7 

-0.1 

0.5 

-0.1 

-0.1 

0.1 

0 

1 

0 

25 

-0.8 

-0.5 

-0.5 

-0.4 

-0.2 

-0.2 

0.4 

-0.3 

0.1 

-7 

0 

-4 

28 

-0.3 

-0.1 

0.0 

-0.4 

-0.2 

0.0 

0.7 

-0.2 

0.2 

1 

-1 

-6 

35 

1.4 

1.1 

1.3 

0.3 

II 

-0.8 

-0.6 

0.8 

0.3 

0.6 

-1 

1 

-9 

38 

1.4 

1.4 

1.8 

0.8 

-1.0 

-0.2 

0.6 

0.4 

0.5 

-2 

2 

-7 

45 

3.1 

2.1 

1.9 

1.3 

-1.4 

0.5 

1.7 

1.8 

1.0 

-9 

2 

-9 

50 

3.9 

1.3 

1.6 

-0.3 

0.2 

-0.4 

2.0 

1.0 

0.9 

-11 

-3 

-10 

55 

4.5 

1.4 

3.1 

0.5 

1.0 

-0.8 

1.0 

-0.2 

1.4 

-15 

0 

-11 

60 

5.4 

3.5 

4.1 

2.0 

2.0 

-0.1 

-2.4 

-1.7 

-1.4 

9 

9 

-5 

70 

9.0 

7.8 

8.2 

7.6 

2.7 

3.3 

-3.0 

-5.0 

-5.2 

59 

28 

34 


Table A.l. Cool season biases for wind variables at XMR as a function of geopotential height (kft). 
See Fig. 3.1 for definitions of verification periods 1, 2, and 3. 

U-wind (m s' 1 ) V-wind (m s' 1 ) Wind Speed (m s' 1 ) Wind Direction (°) 

Verf. Period 123123123123 
Height (kft): 

1 -0.2 0.1 -0.3 "^08 -0^7 01 03 04 08 0 ^5 1 2 

2 -0.9 -0.6 -0.7 -0.6 -0.6 0.2 0.9 0.8 1.2 2 -2 0 

3 -1.3 -0.6 -0.8 0.1 0.1 0.8 0.5 0.3 1.0 2 9 7 

5 -1.1 -0.8 -0.4 1.1 0.1 0.9 -0.5 -0.3 0.5 -2 -3 5 

10 -0.8 -0.4 -0.8 0.0 0.1 -0.4 -0.8 0.0 -0.3 1 5 0 

12 -1.0 -0.7 -0.9 0.0 0.2 -0.6 -0.7 -0.1 -0.4 5 0 3 

20 -1.0 -1.3 -0.9 0.1 0.1 -0.2 -0.5 -0.9 -0.7 -1 0 4 

25 -0.4 -0.6 -0.2 -0.1 0.1 0.5 -0.2 -0.3 0.0 1 -1 2 

28 -0.3 -0.1 -0.5 -0.3 -0.1 0.7 -0.2 0.0 -0.5 1 0 2 

35 0.6 0.7 0.7 -0.6 0.0 0.3 0.5 0.3 0.5 1 0 1 

38 0.7 0.7 0.8 0.1 -0.9 -0.3 0.8 0.1 0.5 -2 1 0 

45 1.0 1.2 0.9 0.2 -0.2 -0.6 1.5 1.5 1.7 -1 0 1 

50 1.9 0.9 0.7 -0.2 0.5 -0.3 1.4 1.1 0.8 0 0 1 

55 3.0 0.2 2.2 -1.1 1.0 -0.4 3.0 0.6 2.1 1 -2 -2 

60 6.8 4.9 6.4 -0.8 1.1 -0.2 5.9 4.3 5.7 -1 -9 -5 

70 10.3 10.1 8.1 3.2 0.2 1.4 7.6 8.1 6.4 8 -6 -6 







Table A.3. Warm season RMS errors for wind variables at XMR as a function of geopotential height 
(kft). See Fig. 3.1 for definitions of verification periods 1, 2, and 3. 


Verf. Period 
Height (kft): 

U-wind (m 
1 2 

s' 1 ) 

3 

V-wind (m 
1 2 

s' 1 ) 

3 

Wind Speed (m s' 1 ) 
12 3 

Wind Direction (°) 
12 3 

1 

2.2 

2.2 

2.7 

2.4 

2.7 

3.0 

1.9 

2.2 

2.7 

51 

52 

50 

2 

2.3 

1.9 

2.6 

2.2 

2.6 

3.1 

2.0 

2.0 

2.9 

45 

49 

52 

3 

2.4 

1.9 

2.8 

2.3 

2.5 

3.0 

2.1 

1.8 

2.9 

47 

57 

55 

5 

3.0 

2.4 

2.8 

2.5 

2.8 

2.9 

2.6 

2.2 

3.0 

52 

55 

63 

10 

2.5 

2.4 

2.8 

2.4 

2.5 

3.0 

2.5 

2.3 

2.9 

49 

48 

48 

12 

2.3 

2.4 

2.8 

2.2 

2.7 

3.0 

2.1 

2.5 

3.0 

47 

47 

54 

20 

2.3 

2.4 

2.8 

2.3 

2.3 

2.5 

2.1 

2.0 

2.3 

49 

53 

50 

25 

2.5 

2.6 

3.2 

2.7 

2.6 

2.5 

2.5 

2.6 

2.8 

44 

43 

50 

28 

2.8 

2.6 

3.6 

3.1 

3.1 

3.2 

3.0 

2.7 

3.2 

39 

40 

47 

35 

3.9 

4.5 

5.0 

3.7 

4.2 

5.1 

3.6 

4.2 

4.8 

42 

43 

49 

38 

4.3 

4.7 

5.7 

3.5 

4.9 

5.1 

4.0 

4.2 

5.1 

36 

42 

49 

45 

5.7 

4.6 

4.6 

4.9 

4.9 

4.9 

5.0 

4.5 

4.5 

42 

36 

37 

50 

5.7 

4.1 

3.8 

3.7 

3.7 

3.7 

4.9 

3.8 

3.3 

43 

47 

43 

55 

5.5 

3.3 

4.0 

3.6 

3.6 

2.8 

3.9 

3.1 

3.2 

60 

51 

48 

60 

5.9 

4.0 

4.8 

3.4 

3.3 

2.5 

4.3 

3.4 

3.3 

76 

58 

62 

70 

9.6 

8.4 

8.8 

8.2 

4.1 

4.6 

5.4 

6.4 

7.0 

85 

78 

80 


Table A.4. Cool season RMS errors for wind variables at XMR as a function of geopotential height 
(kft). See Fig. 3.1 for definitions of verification periods 1, 2, and 3. 


Verf. Period 
Height (kft): 

U-wind (m s' 1 ) 
12 3 

V-wind (m 
1 2 

s' 1 ) 

3 

Wind Speed (m s' 1 ) 
1 2 3 

Wind Direction (°) 
12 3 

1 

2.1 

2.3 

2.5 

2.5 

2.8 

2.7 

2.1 

2.2 

2.2 

30 

39 

35 

2 

2.1 

2.1 

2.5 

2.5 

2.8 

3.0 

2.3 

2.1 

2.6 

37 

39 

39 

3 

2.4 

2.4 

2.9 

2.6 

2.4 

2.9 

2.2 

2.1 

2.8 

40 

46 

40 

5 

2.7 

2.8 

3.2 

3.0 

2.9 

3.3 

2.5 

2.6 

3.1 

36 

43 

46 

10 

2.1 

2.2 

2.7 

2.2 

2.8 

3.0 

2.0 

2.1 

2.8 

38 

48 

46 

12 

2.1 

2.6 

2.7 

2.0 

2.7 

3.0 

2.0 

2.1 

2.6 

37 

37 

45 

20 

2.4 

3.1 

3.2 

2.6 

2.4 

2.9 

2.5 

3.0 

3.0 

19 

19 

28 

25 

2.8 

3.0 

3.5 

2.9 

3.4 

3.7 

2.6 

3.1 

3.1 

14 

15 

22 

28 

3.1 

2.9 

3.4 

3.2 

3.3 

3.4 

2.9 

3.4 

3.4 

15 

13 

19 

35 

3.6 

4.4 

4.9 

4.1 

4.3 

5.3 

3.4 

4.5 

4.5 

11 

13 

16 

38 

3.7 

4.0 

4.6 

4.6 

4.1 

4.6 

3.8 

3.8 

4.4 

13 

13 

17 

45 

4.5 

4.3 

4.3 

5.1 

4.4 

4.6 

4.4 

4.5 

4.9 

12 

9 

9 

50 

4.8 

4.2 

4.6 

4.7 

4.3 

4.9 

4.2 

4.3 

4.7 

12 

11 

12 

55 

4.7 

3.4 

4.2 

3.3 

4.2 

3.5 

4.6 

3.5 

4.4 

15 

18 

14 

60 

7.6 

5.9 

7.2 

3.3 

3.3 

3.6 

6.9 

5.6 

6.7 

37 

37 

37 

70 

10.8 

10.5 

8.7 

4.4 

3.6 

4.5 

8.9 

9.4 

8.0 

72 

65 

66 
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Table A.5. Warm season biases for wind variables at EDW as a function of geopotential height (kft). 


Verf. Period 
Height (kft): 

U-wind (m s' 1 ) 
12 3 

1 

— 

— 

— 

2 

— 

— 

— 

3 

— 

— 

— 

5 

-0.6 

0.3 

-1.0 

10 

-0.6 

0.0 

-0.5 

12 

-0.8 

-0.3 

-0.7 

20 

-0.9 

-1.2 

-2.1 

25 

-0.9 

-1.8 

-1.6 

28 

-1.2 

-1.5 

-1.9 

35 

-0.1 

-0.6 

-1.2 

38 

0.2 

-0.6 

-1.3 

45 

-0.8 

-0.4 

-1.0 

50 

-0.5 

-0.5 

-0.5 

55 

1.3 

1.5 

2.2 

60 

2.5 

2.7 

3.6 

70 

5.9 

3.7 

5.2 


V-wind (m s' 1 ) Wind Speed (m s' 1 ) Wind Direction (°) 



1.3 -0.9 


0.8 -0.5 

0.5 0.1 

1.1 -0.5 

1.2 - 0.2 


-0.3 
-1.2 -1.4 

-0.9 -0.8 

- 0.2 0.1 

-0.9 -1.1 

-0.7 -1.4 


1.2 0.1 -0.3 -1.0 
0.1 0.3 0.3 -1.1 
0.0 -0.5 -1.0 -1.5 
0.6 -0.8 -0.5 -0.3 
0.7 0.2 0.1 1.0 
0.3 -1.3 0.0 0.6 


5.2 -0.5 -0.9 -0.1 -2.1 -1.2 -1.8 



Table A.6. Cool season biases for wind variables at EDW as a function of geopotential height (kft). 


Verf. Period 
Height (kft): 

U-wind (m 
1 2 

s' 1 ) 

3 

V-wind (m s' 1 ) 

1 2 3 

Wind Speed (m s' 1 ) 
12 3 

Wind Direction (°) 
12 3 

1 

2 

3 

5 

-1.4 

-2.5 

-3.6 

-0.1 

-0.2 

-0.3 

-0.2 

-0.4 

0.1 

-8 

3 

5 

10 

-1.1 

-2.7 

-3.0 

0.9 

0.7 

0.4 

-0.6 

-0.6 

-1.2 

-7 

-2 

-1 

12 

-1.2 

-1.9 

-2.4 

1.1 

0.5 

1.0 

-0.6 

-0.6 

-1.6 

2 

5 

4 

20 

0.9 

0.9 

-0.6 

0.8 

1.3 

1.9 

0.6 

-0.3 

-1.2 

-6 

-9 

-10 

25 

-0.1 

0.1 

-0.6 

1.1 

1.8 

2.5 

-1.0 

-0.5 

-2.0 

-ii 

-10 

-9 

28 

-0.2 

1.0 

0.3 

1.2 

1.2 

2.1 

0.2 

0.6 

-0.2 

-6 

-6 

-6 

35 

1.7 

2.4 

1.4 

0.9 

2.8 

4.4 

0.6 

-0.2 

-1.5 

-6 

-6 

-9 

38 

0.0 

0.7 

1.0 

0.2 

1.3 

2.9 

-1.5 

-0.7 

-2.0 

-2 

-5 

-8 

45 

1.5 

2.5 

2.2 

-2.6 

0.5 

0.5 

2.1 

1.2 

1.2 

5 

-3 

-3 

50 

-0.5 

1.7 

1.6 

1.3 

3.1 

1.7 

-1.3 

0.4 

0.4 

-3 

-8 

-5 

55 

0.9 

1.7 

2.7 

0.8 

0.8 

2.0 

0.4 

1.2 

2.1 

-7 

-5 

-5 

60 

1.2 

3.4 

4.9 

-0.4 

0.6 

1.8 

1.3 

2.5 

3.9 

-1 

-3 

-10 

70 

4.1 

5.4 

4.7 

-2.4 

-1.9 

-3.1 

4.7 

5.4 

3.9 

-2 

7 

-3 
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(kft). See Fig. 3.1 for definitions of verification periods 1, 

2, anc 

13. 




U-wind (m s' 1 ) 

V-wind (m 

s’ 1 ) 

Wind Speed (m s' 1 ) 

Wind Direction (°) 

Verf. Period 
Height (kft): 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

1 

2 

3 

5 

4.0 

4.2 

3.6 

2.6 

2.7 

3.4 

3.7 

3.7 

3.7 

35 

42 

36 

10 

2.6 

3.1 

3.2 

2.4 

2.9 

3.4 

2.4 

2.9 

3.4 

38 

48 

44 

12 

2.6 

3.3 

3.5 

2.7 

3.0 

3.3 

2.2 

3.0 

3.7 

37 

42 

41 

20 

2.9 

3.2 

4.4 

3.1 

3.6 

3.9 

2.9 

3.3 

3.6 

27 

32 

42 

25 

3.1 

4.1 

4.7 

3.3 

3.0 

3.6 

3.2 

3.4 

4.6 

28 

30 

36 

28 

2.9 

3.4 

4.4 

3.4 

3.9 

4.1 

2.6 

3.1 

4.2 

19 

26 

35 

35 

3.4 

3.7 

4.2 

3.4 

4.2 

4.6 

3.6 

3.4 

3.9 

14 

24 

26 

38 

3.4 

4.0 

4.3 

3.3 

3.6 

4.8 

3.4 

3.9 

4.1 

15 

18 

21 

45 

3.6 

3.4 

4.8 

4.6 

4.0 

4.2 

4.1 

3.8 

4.6 

20 

19 

22 

50 

3.2 

3.9 

4.6 

4.1 

3.7 

4.5 

3.9 

4.0 

4.5 

28 

19 

29 

55 

3.1 

3.6 

4.3 

4.0 

3.4 

3.1 

2.9 

3.4 

3.9 

43 

35 

29 

60 

4.1 

4.5 

5.0 

3.2 

3.6 

3.3 

3.1 

3.4 

4.1 

54 

54 

58 

70 

6.5 

4.7 

5.9 

3.4 

3.1 

2.3 

5.7 

4.0 

4.5 

61 

56 

57 


Table A.8. Cool season RMS errors for wind variables at EDW as a function of geopotential height 
(kft). See Fig. 3.1 for definitions of verification periods 1, 2, and 3. 


Verf. Period 
Height (kft): 

U-wind (m 
1 2 

s' 1 ) 

3 

V-wind (m s' 1 ) 

1 2 3 

Wind Speed (m s' 1 ) 
1 2 3 

Wind Direction (°) 
12 3 

1 

2 

3 

5 

3.8 

4.6 

5.2 

3.4 

3.3 

3.6 

2.8 

3.7 

3.6 

53 

60 

62 

10 

2.4 

3.8 

4.2 

3.3 

3.6 

3.4 

2.8 

3.2 

3.7 

28 

31 

44 

12 

3.1 

3.5 

4.4 

4.0 

4.3 

4.1 

3.1 

3.5 

4.2 

29 

32 

38 

20 

4.1 

4.5 

4.3 

4.0 

5.8 

5.6 

3.5 

5.5 

5.3 

20 

28 

27 

25 

4.9 

5.6 

6.0 

4.4 

5.3 

5.9 

4.3 

4.7 

5.9 

30 

24 

25 

28 

5.0 

4.9 

5.3 

5.0 

5.4 

5.9 

4.5 

5.0 

4.7 

15 

19 

18 

35 

5.2 

5.9 

6.1 

5.2 

6.6 

8.1 

4.8 

5.2 

6.2 

13 

14 

17 

38 

5.2 

5.5 

6.4 

6.3 

5.9 

7.5 

6.2 

6.0 

6.8 

15 

17 

19 

45 

6.0 

5.8 

6.0 

5.7 

5.7 

4.9 

5.8 

5.1 

5.1 

19 

20 

21 

50 

4.2 

4.0 

3.6 

5.1 

5.6 

5.2 

4.7 

3.9 

3.7 

18 

19 

22 

55 

3.5 

3.4 

4.2 

4.3 

4.4 

4.6 

4.0 

3.7 

4.3 

20 

18 

27 

60 

4.7 

5.9 

6.8 

3.4 

3.6 

3.6 

5.2 

5.2 

5.8 

19 

20 

22 

70 

5.8 

6.8 

5.4 

3.7 

4.2 

4.7 

6.8 

6.8 

5.5 

34 

26 

36 
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Table A.9. Sample sizes of wind data used to calculate bias and RMS errors at XMR and 
EDW during the warm and cool seasons as a function of geopotential height (kft). See Fig. 
3.1 for definitions of verification periods 1, 2, and 3. 


Verf. Period 
Height (kft): 

XMR (warm) 

1 2 3 

XMR (cool) 

1 2 3 

EDW (warm) 
12 3 

EDW (cool) 

1 2 3 

i 

EB 

127 

127 

125 

124 

130 

— 

— 

— 

— 

— 

— 

2 

m 

127 

128 

121 

124 

130 

— 

— 

— 

— 

— 

— 

3 

m 

130 

131 

121 

124 

129 

— 

— 

— 

— 

— 

— 

5 

135 

132 

128 

128 

125 

130 

78 

76 

75 

57 

58 

57 

10 

136 

132 

134 

126 

125 

126 

78 

76 

76 

55 

57 

55 

12 

133 

133 

131 

125 

128 

128 

78 

74 

74 

56 

53 

55 * 

20 

133 

134 

126 

126 

126 

129 

73 

74 

73 

56 

56 

56 

25 

134 

132 

131 

127 

128 

129 

72 

71 

72 

55 

58 

55 

28 

133 

131 

131 

127 

125 

129 

74 

70 

72 

55 

55 

53 

35 

136 

134 

130 

128 

127 

130 

72 

72 

72 

51 

55 

53 

38 

136 

131 

129 

124 

126 

128 

72 

69 

73 

54 

57 

54 

45 

132 

132 

128 

125 

128 

126 

71 

70 

72 

53 

55 

52 

50 

136 

134 

124 

122 

122 

127 

65 

70 

70 

53 

54 

53 

55 

120 

115 

109 

98 

90 

94 

58 

60 

61 

40 

44 

42 

60 

120 

113 

112 

93 

90 

93 

60 

61 

58 

40 

42 

42 

70 

117 

111 

106 

92 

86 

87 

37 

37 

37 

26 

26 

26 
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of fully informing the reader of the resources used to conduct the work reported herein. 
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